file

Synchronously checks if any of the files provided do not exist.

For example:

//-- these exist
// ./data/credentials.env
// ./data/results.json

if (!utils.file.checkFile('./data/results.json')) {
   //-- retrieve the results
   utils.ijs.await(async($$, console) => {
     results = await connection.query('SELECT XYZ from Contacts');
     utils.file.write('./data/results.json', results);
   });
} else {
   results = utils.file.readJSON('./data/results.json');
}

Note, you can also ask for multiple files at once

utils.file.checkFile(
   './data/credentials.env',
   './data/results.json',
   './data/results.csv'
);
// false

or as an array:

utils.file.checkFile(['./data/credentails.env']);
// true

Checks if a single file exists

List files in a directory

Finds files in a directory, returning only the file names and paths of those that match a function.

Note the matching function passes both fileNames and DirEnt objects
{(fileName:String, file:DirEnt) => Boolean}
allowing for checking for files:.isFile(), directories:.isDirectory(), symbolic links:.isSymbolicLink(), etc.

For example, if there is a ./tmp folder, with:

• ./tmp/fileA (file)
• ./tmp/fileB (file)
• ./tmp/dirA (directory)
• ./tmp/dirB (directory)

You could find only files like the following:

utils.file.matchFiles('./tmp', (fileName, file) => file.isFile());
// ['./tmp/fileA', './tmp/fileB'];

or find directories ending with the letter B:

utils.file.matchFiles('./tmp',
 (fileName, file) => file.isDirectory() && fileName.endsWith('B')
);
// ['./tmp/dirB'];

Note: passing false as the last parameter will only return file names

utils.file.matchFiles('./tmp', (fileName) => fileName.startsWith('file'), false);
// ['fileA', 'fileB']

List the current path (working directory)

Reads a file in as text.

This can be handy for tinkering / cleaning of small sets of data.

Note that this uses utf-8 by default for the encoding

Read JSON file.

Note that this uses 'utf-8' encoding by default

For very long or time-intensive executions, sometimes it is better to cache the results than to execute them every single time.

Note that this works synchronously, and can be easier to use than if promises are involved.

As opposed to ijs.useCache - which works with promises.

shouldWrite = true; /// we will write to the cache with the results from the execution
expensiveResults = utils.file.useCache(shouldWrite, './cache', 'expensive.json', () => {
   const data = d3.csvParse(utils.file.readFile('./someFile.csv'))
     .map(obj => ({ ...obj, date: Date.parse(obj.epoch) }));
   
   const earliestDate = utils.date.startOfDay( utils.agg.min(data, 'date') );
   const lastDate = utils.date.endOfDay( utils.agg.max(data, 'date') );

   // binning or lots of other things.

   return finalResults;
});

expensiveresults.length = 1023424;

but sometimes I would rather just skip to the end

shouldWrite = false; /// we will read from the cache instead,
// everything else remains the same

expensiveResults = utils.file.useCache(shouldWrite, './cache', 'expensive.json', () => {
   const data = d3.csvParse(utils.file.readFile('./someFile.csv'))
     .map(obj => ({ ...obj, date: Date.parse(obj.epoch) }));

   //-- function can remain untouched,
   //-- BUT nothing in here will be executed
   //-- since we are reading from the cache
});

//-- completely transparent to the runner
expensiveresults.length = 1023424;

Writes to a file

Note that this uses utf-8 as the encoding by default

const myString = `hello`;
utils.file.writeFile('./tmp', myString);
const newString = utils.file.readFile('./tmp');
newString; // 'hello';

Note, you can append to the file by passing {append:true} in the options.

Writes to a file

NOTE that this uses utf-8 as the default encoding

const weather = [
  { id: 1, city: 'Seattle',  month: 'Aug', precip: 0.87 },
  { id: 0, city: 'Seattle',  month: 'Apr', precip: 2.68 },
  { id: 2, city: 'Seattle',  month: 'Dec', precip: 5.31 },
  { id: 3, city: 'New York', month: 'Apr', precip: 3.94 },
  { id: 4, city: 'New York', month: 'Aug', precip: 4.13 },
  { id: 5, city: 'New York', month: 'Dec', precip: 3.58 },
  { id: 6, city: 'Chicago',  month: 'Apr', precip: 3.62 },
  { id: 8, city: 'Chicago',  month: 'Dec', precip: 2.56 },
  { id: 7, city: 'Chicago',  month: 'Aug', precip: 3.98 }
];
utils.file.writeJSON('./data/weather.json', weather);

const myWeather = utils.file.readJSON('./data/weather.json');
myWeather.length; // 9

Note, passing append:true in the options, will let you append text before writing, useful for dealing with large and complex files.

weatherEntry1 = { id: 1, city: 'Seattle',  month: 'Aug', precip: 0.87 };
weatherEntry2 = { id: 0, city: 'Seattle',  month: 'Apr', precip: 2.68 };
weatherEntry3 = { id: 2, city: 'Seattle',  month: 'Dec', precip: 5.31 };

utils.file.writeJSON('./data/weather2.json', weatherEntry1, { prefix: '[' });
utils.file.writeJSON('./data/weather2.json', weatherEntry2, { append: true, prefix: ', ' });
utils.file.writeJSON('./data/weather2.json', weatherEntry3, { append: true, prefix: ', ', suffix: ']' });

utils.file.readJSON('./data/weather.json');

//-- single line shown here on multiple lines for clarity
// [{"id":1,"city":"Seattle","month":"Aug","precip":0.87}
// ,{"id":0,"city":"Seattle","month":"Apr","precip":2.68}
// ,{"id":2,"city":"Seattle","month":"Dec","precip":5.31}]