<?xml version="1.0" encoding="UTF-8"?><rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	>

<channel>
	<title>csv | Nerdpress.org</title>
	<atom:link href="https://nerdpress.org/tag/csv/feed/" rel="self" type="application/rss+xml" />
	<link>https://nerdpress.org</link>
	<description>...dev, tech problems and solutions.</description>
	<lastBuildDate>Thu, 19 May 2022 07:57:09 +0000</lastBuildDate>
	<language>en-US</language>
	<sy:updatePeriod>
	hourly	</sy:updatePeriod>
	<sy:updateFrequency>
	1	</sy:updateFrequency>
	<generator>https://wordpress.org/?v=6.9.4</generator>
	<item>
		<title>Filter a csv file with nodejs streams</title>
		<link>https://nerdpress.org/2022/01/17/filter-a-csv-file-with-nodejs-streams/</link>
		
		<dc:creator><![CDATA[Ivo Bathke]]></dc:creator>
		<pubDate>Mon, 17 Jan 2022 08:50:22 +0000</pubDate>
				<category><![CDATA[Uncategorized]]></category>
		<category><![CDATA[csv]]></category>
		<category><![CDATA[nodejs]]></category>
		<category><![CDATA[streams]]></category>
		<guid isPermaLink="false">https://nerdpress.org/?p=3103</guid>

					<description><![CDATA[<p>So we have a CSV file with a member list and we want to filter all empty emails. FirstName,LastName,Email Max,Mustermann,max@mustermann.de Maxi,Hasnoemail, With nodejs streams this is almost a one liner. We will use the csv package from the CSV project which has some sophisticated packages to parse and transform csv data with nodejs streams. We &#8230; </p>
<p class="link-more"><a href="https://nerdpress.org/2022/01/17/filter-a-csv-file-with-nodejs-streams/" class="more-link">Continue reading<span class="screen-reader-text"> "Filter a csv file with nodejs streams"</span></a></p>
The post <a href="https://nerdpress.org/2022/01/17/filter-a-csv-file-with-nodejs-streams/">Filter a csv file with nodejs streams</a> first appeared on <a href="https://nerdpress.org">Nerdpress.org</a>.]]></description>
										<content:encoded><![CDATA[<p>So we have a CSV file with a member list and we want to filter all empty emails.</p>



<pre class="wp-block-preformatted">FirstName,LastName,Email
Max,Mustermann,max@mustermann.de
Maxi,Hasnoemail,</pre>



<p>With nodejs streams this is almost a one liner.</p>



<p>We will use the <a href="https://csv.js.org/">csv</a> package from the CSV project which has some sophisticated packages to parse and transform csv data with nodejs streams.</p>


<div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate">
npm install csv
</pre></div>


<p>We will further use node as ESM (ECMAScript modules) to be shiningly modern and so lets create a file: <code>index.mjs</code> <br /><em>Note the .mjs extension, which will tell node to interprete this as ESM. (Since nodejs v12.20.0 &amp;&amp; v14.13.0)</em></p>



<p>We import the packages and create the filesystem streams to read the file, then built a pipeline with the streams and the single steps to process the data and write the results into a new file.<br />Ok let&#8217;s go:</p>



<span id="more-3103"></span>


<div class="wp-block-syntaxhighlighter-code "><pre class="brush: jscript; title: ; notranslate">
import {pipeline} from &#039;stream&#039;;
import {parse, transform, stringify} from &#039;csv&#039;;
import {createReadStream, createWriteStream} from &#039;fs&#039;;

pipeline(
  createReadStream(`./members.csv`),
  parse(),
  transform(data =&gt; data&#x5B;2] !== &#039;&#039; ? data : null),
  stringify({}),
  createWriteStream(`./members_filtered.csv`),
  err =&gt; err ? console.error(&#039;Pipeline failed.&#039;, err) : console.log(&#039;Pipeline succeeded.&#039;)
);

</pre></div>


<p>That is all, folks! Here is the <a href="https://gist.github.com/ivoba/29fc74ecd8ee6c2344c89d75418836e8">gist</a>.</p>



<p>We now have filtered all rows where the email is empty and saved the remaining rows to members_filtered.csv file.</p>



<p>We also could have done this with only pipes but this is considered unsafe (<a href="https://stackoverflow.com/questions/58875655/whats-the-difference-between-pipe-and-pipeline-on-streams/60459320#60459320">https://stackoverflow.com/questions/58875655/whats-the-difference-between-pipe-and-pipeline-on-streams/60459320#60459320</a>) since pipeline is an optimized stream handling with better cleanups and error handling.<br />Nevertheless for the sake of completeness, the pipes version:</p>


<div class="wp-block-syntaxhighlighter-code "><pre class="brush: jscript; title: ; notranslate">
import {parse, transform, stringify} from &#039;csv&#039;;
import {createReadStream, createWriteStream} from &#039;fs&#039;;

createReadStream(`./members.csv`)
  .pipe(parse())
  .pipe(transform(data =&gt; data&#x5B;2] !== &#039;&#039; ? data : null))
  .pipe(stringify({}))
  .pipe(createWriteStream(`./members_filtered.csv`))
  .on(&#039;finish&#039;, data =&gt; console.log(&#039;Done&#039;));
</pre></div>The post <a href="https://nerdpress.org/2022/01/17/filter-a-csv-file-with-nodejs-streams/">Filter a csv file with nodejs streams</a> first appeared on <a href="https://nerdpress.org">Nerdpress.org</a>.]]></content:encoded>
					
		
		
			</item>
	</channel>
</rss>
