Jupyter-IJavaScript-Utils

Simple file for working with files, (or storing and loading json data)

Writing files
- writeFile(path, string) - write or append to file with plain text
- writeJSON(path, any) - write or append to a file with objects converted to JSON
reading files
- readFile(path, string) - read a file as plain text
- readJSON(path, any) - read data as JSON
listing directory
- pwd() - list the current path
- listFiles(path) - list files in a diven path
- matchFiles(path, matchingFn) - find files or directories based on type of file or name
checking files exist
- checkFile(...paths) - check if a file at a path exists

For example, we just generated a dataset we want to come back to.

const weather = [
  { id: 1, city: 'Seattle',  month: 'Aug', precip: 0.87 },
  { id: 0, city: 'Seattle',  month: 'Apr', precip: 2.68 },
  { id: 2, city: 'Seattle',  month: 'Dec', precip: 5.31 },
  { id: 3, city: 'New York', month: 'Apr', precip: 3.94 },
  { id: 4, city: 'New York', month: 'Aug', precip: 4.13 },
  { id: 5, city: 'New York', month: 'Dec', precip: 3.58 },
  { id: 6, city: 'Chicago',  month: 'Apr', precip: 3.62 },
  { id: 8, city: 'Chicago',  month: 'Dec', precip: 2.56 },
  { id: 7, city: 'Chicago',  month: 'Aug', precip: 3.98 }
];
utils.file.writeJSON('./data/weather.json', weather);

... Later on

I forgot which directory that the notebook is running in:

utils.file.pwd();
// /Users/path/to/notebook/

Now, I'd like to look at the files I currently have saved:

utils.file.listFiles('.');
// [ 'data', 'package.json', ... ]

utils.file.listFiles('./data');
// ['weather.json', 'barley.json', 'cars.json']

Great! we can load in the data

data = utils.file.loadJSON('./data/weather.json');
// -- data already deserialized
data.length
// 9

... continue massaging the data as we wanted.

Methods

(static) checkFile(…files) → {Array.<String>}

Synchronously checks if any of the files provided do not exist.

For example:

//-- these exist
// ./data/credentials.env
// ./data/results.json

if (!utils.file.checkFile('./data/results.json')) {
   //-- retrieve the results
   utils.ijs.await(async($$, console) => {
     results = await connection.query('SELECT XYZ from Contacts');
     utils.file.write('./data/results.json', results);
   });
} else {
   results = utils.file.readJSON('./data/results.json');
}

Note, you can also ask for multiple files at once

utils.file.checkFile(
   './data/credentials.env',
   './data/results.json',
   './data/results.csv'
);
// false

or as an array:

utils.file.checkFile(['./data/credentails.env']);
// true

Parameters:

Name	Type	Attributes	Description
`files`	String	<repeatable>	List of file paths to check (can use relative paths, like './') see listFiles() or pwd() to help you)

Returns:

null if all files are found, or array of string paths of files not found

Type: Array.<String>

(static) listFiles(directoryPath, readdirOptionsopt)

See:

pwd() - to get the current working directory
matchFiles(path, matchingFn) - find files or directories based on type of file or name

List files in a directory

Example

utils.file.listFiles('./');
// ['.gitignore', 'data', ... ];

Parameters:

Name	Type	Attributes	Default	Description
`directoryPath`	String			path of the directory to list
`readdirOptions`	Object	<optional>	`null`	object with options to pass to fs readdir

(static) matchFiles(directoryPath, matchingFunction, returnFullPathopt) → {Array.<String>}

See:

listFiles(path) - list files in a diven path

Finds files in a directory, returning only the file names and paths of those that match a function.

Note the matching function passes both fileNames and DirEnt objects
{(fileName:String, file:DirEnt) => Boolean}
allowing for checking for files:.isFile(), directories:.isDirectory(), symbolic links:.isSymbolicLink(), etc.

For example, if there is a ./tmp folder, with:

./tmp/fileA (file)
./tmp/fileB (file)
./tmp/dirA (directory)
./tmp/dirB (directory)

You could find only files like the following:

utils.file.matchFiles('./tmp', (fileName, file) => file.isFile());
// ['./tmp/fileA', './tmp/fileB'];

or find directories ending with the letter B:

utils.file.matchFiles('./tmp',
 (fileName, file) => file.isDirectory() && fileName.endsWith('B')
);
// ['./tmp/dirB'];

Note: passing false as the last parameter will only return file names

utils.file.matchFiles('./tmp', (fileName) => fileName.startsWith('file'), false);
// ['fileA', 'fileB']

Parameters:

Name	Type	Attributes	Default	Description
`directoryPath`	String			path of the directory to match within
`matchingFunction`	function			(DirEnt) => Boolean function to determine if the path should be returned or not
`returnFullPath`	Boolean	<optional>	`true`	whether the full path should be returned

Returns:

list of the files that match

Type: Array.<String>

(static) pwd() → {string}

See:

listFiles(path) - to list the files of a directory

List the current path (working directory)

Example

utils.file.pwd(); // /user/path/to/notebook

Returns:

Type: string

(static) readFile(filePath, fsOptions) → {String}

See:

writeFile(filePath, contents, fsOptions) - for writing

Reads a file in as text.

This can be handy for tinkering / cleaning of small sets of data.

Note that this uses utf-8 by default for the encoding

Example

sillySong = utils.file.load('../data/pirates.txt');

sillySong.split(/\n[ \t]*\n/)        // split on multiple line breaks
  .map(stanza => stanza.split(/\n/)  // split lines by newline
    .map(line => line.trim())        // trim each line
  );
sillySong[0][0]; // I am the very model of a modern Major-General,

Parameters:

Name	Type	Description
`filePath`	String	path of the file to load
`fsOptions`	Object	options to pass for fsRead (ex: { encoding: 'utf-8' })

Returns:

Type: String

(static) readJSON(filePath, fsOptions)

See:

writeJSON(path, data, fsOptions) - to write the data

Read JSON file.

Note that this uses 'utf-8' encoding by default

Example

const weather = [
  { id: 1, city: 'Seattle',  month: 'Aug', precip: 0.87 },
  { id: 0, city: 'Seattle',  month: 'Apr', precip: 2.68 },
  { id: 2, city: 'Seattle',  month: 'Dec', precip: 5.31 },
  { id: 3, city: 'New York', month: 'Apr', precip: 3.94 },
  { id: 4, city: 'New York', month: 'Aug', precip: 4.13 },
  { id: 5, city: 'New York', month: 'Dec', precip: 3.58 },
  { id: 6, city: 'Chicago',  month: 'Apr', precip: 3.62 },
  { id: 8, city: 'Chicago',  month: 'Dec', precip: 2.56 },
  { id: 7, city: 'Chicago',  month: 'Aug', precip: 3.98 }
];
utils.file.writeJSON('./data/weather.json', weather);

const myWeather = utils.file.readJSON('./data/weather.json');
myWeather.length; // 9

Parameters:

Name	Type	Description
`filePath`	string	path of the file to load
`fsOptions`	Object	options to pass for fsRead (ex: { encoding: 'utf-8' })

(static) writeFile(filePath, contents, fsOptions)

See:

readFile(filePath, fsOptions) - for reading

Writes to a file

Note that this uses utf-8 as the encoding by default

const myString = `hello`;
utils.file.writeFile('./tmp', myString);
const newString = utils.file.readFile('./tmp');
newString; // 'hello';

Note, you can append to the file by passing {append:true} in the options.

Parameters:

Name Type Description

filePath

string

path of the file to write

contents

string

contents of the file

fsOptions

Object

nodejs fs writeFileSync, appendFileSync options

Properties

Name	Type	Description
`append`	Boolean	if true, will append the text to the file
`encoding`	String	encoding to use when writing the file.

(static) writeJSON(filePath, contents, fsOptions)

See:

readJSON(filePath, fsOptions) - for reading

Writes to a file

NOTE that this uses utf-8 as the default encoding

const weather = [
  { id: 1, city: 'Seattle',  month: 'Aug', precip: 0.87 },
  { id: 0, city: 'Seattle',  month: 'Apr', precip: 2.68 },
  { id: 2, city: 'Seattle',  month: 'Dec', precip: 5.31 },
  { id: 3, city: 'New York', month: 'Apr', precip: 3.94 },
  { id: 4, city: 'New York', month: 'Aug', precip: 4.13 },
  { id: 5, city: 'New York', month: 'Dec', precip: 3.58 },
  { id: 6, city: 'Chicago',  month: 'Apr', precip: 3.62 },
  { id: 8, city: 'Chicago',  month: 'Dec', precip: 2.56 },
  { id: 7, city: 'Chicago',  month: 'Aug', precip: 3.98 }
];
utils.file.writeJSON('./data/weather.json', weather);

const myWeather = utils.file.readJSON('./data/weather.json');
myWeather.length; // 9

Note, passing append:true in the options, will let you append text before writing, useful for dealing with large and complex files.

weatherEntry1 = { id: 1, city: 'Seattle',  month: 'Aug', precip: 0.87 };
weatherEntry2 = { id: 0, city: 'Seattle',  month: 'Apr', precip: 2.68 };
weatherEntry3 = { id: 2, city: 'Seattle',  month: 'Dec', precip: 5.31 };

utils.file.writeJSON('./data/weather2.json', weatherEntry1, { prefix: '[' });
utils.file.writeJSON('./data/weather2.json', weatherEntry2, { append: true, prefix: ', ' });
utils.file.writeJSON('./data/weather2.json', weatherEntry3, { append: true, prefix: ', ', suffix: ']' });

utils.file.readJSON('./data/weather.json');

//-- single line shown here on multiple lines for clarity
// [{"id":1,"city":"Seattle","month":"Aug","precip":0.87}
// ,{"id":0,"city":"Seattle","month":"Apr","precip":2.68}
// ,{"id":2,"city":"Seattle","month":"Dec","precip":5.31}]

Parameters:

Name Type Description

filePath

string

path of the file to write

contents

string

contents of the file

fsOptions

Object

nodejs fs writeFileSync, appendFileSync options

Properties

Name	Type	Description
`append`	Boolean	if true, will append the text to the file
`prefix`	Boolean	string to add before writing the json, like an opening bracket '[' or comma ','
`prefix`	Boolean	string to add before writing the json, like a closing bracket ']'
`encoding`	String	encoding to use when writing the file.