Filter a csv file with nodejs streams

So we have a CSV file with a member list and we want to filter all empty emails.


With nodejs streams this is almost a one liner.

We will use the csv package from the CSV project which has some sophisticated packages to parse and transform csv data with nodejs streams.

npm install csv

We will further use node as ESM (ECMAScript modules) to be shiningly modern and so lets create a file: index.mjs
Note the .mjs extension, which will tell node to interprete this as ESM. (Since nodejs v12.20.0 && v14.13.0)

We import the packages and create the filesystem streams to read the file, then built a pipeline with the streams and the single steps to process the data and write the results into a new file.
Ok let’s go:

import {pipeline} from 'stream';
import {parse, transform, stringify} from 'csv';
import {createReadStream, createWriteStream} from 'fs';

  transform(data => data[2] !== '' ? data : null),
  err => err ? console.error('Pipeline failed.', err) : console.log('Pipeline succeeded.')

That is all, folks! Here is the gist.

We now have filtered all rows where the email is empty and saved the remaining rows to members_filtered.csv file.

We also could have done this with only pipes but this is considered unsafe ( since pipeline is an optimized stream handling with better cleanups and error handling.
Nevertheless for the sake of completeness, the pipes version:

import {parse, transform, stringify} from 'csv';
import {createReadStream, createWriteStream} from 'fs';

  .pipe(transform(data => data[2] !== '' ? data : null))
  .on('finish', data => console.log('Done'));