sciris.sc_fileio

Functions for reading/writing to files, including pickles, JSONs, and Excel.

Highlights:

• save() / load(): efficiently save/load any Python object (via pickling)

• savetext() / loadtext(): likewise, for text

• savejson() / loadjson(): likewise, for JSONs

• saveyaml() / saveyaml(): likewise, for YAML

• thisdir(): get current folder

• getfilelist(): easy way to access glob

• rmpath(): remove files and folders

Functions

dumpstr	Dump an object as a bytes-like string (rarely used); see sc.loadstr()
getfilelist	A shortcut for using glob.
ispath	Alias to isinstance(obj, Path).
jsonify	This is the main conversion function for Python data-structures into JSON-compatible data structures (note: sc.sanitizejson()/sc.jsonify() are identical).
jsonpickle	Save any Python object to a JSON file using jsonpickle.
jsonunpickle	Use jsonpickle to restore an object (see jsonpickle())
load	Load a file that has been saved as a gzipped pickle file, e.g.
loadjson	Convenience function for reading a JSON file (or string).
loadobj	Load a file that has been saved as a gzipped pickle file, e.g.
loadspreadsheet	Load a spreadsheet as a dataframe or a list of lists.
loadstr	Like loadobj(), but for a bytes-like string (rarely used).
loadtext	Convenience function for reading a text file
loadyaml	Convenience function for reading a YAML file (or string).
loadzip	Convenience function for reading a zip file
makefailed	Create a class -- not an object! -- that contains the failure info for a pickle that failed to load
makefilepath	Utility for taking a filename and folder -- or not -- and generating a valid path from them.
path	Alias to pathlib.Path() with some additional input sanitization:
rmpath	Remove file(s) and folder(s).
sanitizefilename	Takes a potentially Linux- and Windows-unfriendly candidate file name, and returns a "sanitized" version that is more usable.
sanitizejson	This is the main conversion function for Python data-structures into JSON-compatible data structures (note: sc.sanitizejson()/sc.jsonify() are identical).
save	Save an object to file as a gzipped pickle -- use compression 5 by default, since more is much slower but not much smaller.
savejson	Convenience function for saving to a JSON file.
saveobj	Save an object to file as a gzipped pickle -- use compression 5 by default, since more is much slower but not much smaller.
savespreadsheet	Semi-simple function to save data nicely to Excel.
savetext	Convenience function for saving a text file -- accepts a string or list of strings; can also save an arbitrary object, in which case it will first convert to a string.
saveyaml	Convenience function for saving to a YAML file.
savezip	Create a zip file from the supplied list of files (or less commonly, supplied data)
thisdir	Tiny helper function to get the folder for a file, usually the current file.

Classes

Blobject	A wrapper for a binary file -- rarely used directly.
Empty	Another empty class to represent a failed object loading, but do not proceed with setstate
Failed	An empty class to represent a failed object loading
Spreadsheet	A class for reading and writing Excel files in binary format.
UniversalFailed	A universal failed object class, that preserves as much data as possible

load(filename=None, folder=None, verbose=False, die=None, remapping=None, method='pickle', **kwargs)

Load a file that has been saved as a gzipped pickle file, e.g. by sc.save(). Accepts either a filename (standard usage) or a file object as the first argument. Note that sc.load()/sc.loadobj() are aliases of each other.

Note: be careful when loading pickle files, since a malicious pickle can be used to execute arbitrary code.

When a pickle file is loaded, Python imports any modules that are referenced in it. This is a problem if module has been renamed. In this case, you can use the remapping argument to point to the new modules or classes.

Parameters

• filename (str/Path) – the filename (or full path) to load

• folder (str/Path) – the folder

• verbose (bool) – print details

• die (bool) – whether to raise an exception if errors are encountered (otherwise, load as much as possible)

• remapping (dict) – way of mapping old/unavailable module names to new

• method (str) – method for loading (usually pickle or dill)

• kwargs (dict) – passed to pickle.loads()/dill.loads()

Examples:

obj = sc.load('myfile.obj') # Standard usage
old = sc.load('my-old-file.obj', method='dill', ignore=True) # Load classes from saved files
old = sc.load('my-old-file.obj', remapping={'foo.Bar':cat.Mat}) # If loading a saved object containing a reference to foo.Bar that is now cat.Mat
old = sc.load('my-old-file.obj', remapping={'foo.Bar':('cat', 'Mat')}) # Equivalent to the above

New in version 1.1.0: “remapping” argument

New in version 1.2.2: ability to load non-gzipped pickles; support for dill; arguments passed to loader

save(filename=None, obj=None, compresslevel=5, verbose=0, folder=None, method='pickle', sanitizepath=True, die=False, *args, **kwargs)

Save an object to file as a gzipped pickle – use compression 5 by default, since more is much slower but not much smaller. Once saved, can be loaded with sc.load(). Note that sc.save()/sc.saveobj() are identical.

Parameters

• filename (str/Path) – the filename to save to; if str, passed to sc.makefilepath()

• obj (anything) – the object to save

• compresslevel (int) – the level of gzip compression

• verbose (int) – detail to print

• folder (str) – passed to sc.makefilepath()

• method (str) – whether to use pickle (default) or dill

• die (bool) – whether to fail if the object can’t be pickled (else, try dill)

• sanitizepath (bool) – whether to sanitize the path prior to saving

• args (list) – passed to pickle.dumps()

• kwargs (dict) – passed to pickle.dumps()

Example:

myobj = ['this', 'is', 'a', 'weird', {'object':44}]
sc.save('myfile.obj', myobj)
sc.save('myfile.obj', myobj, method='dill') # Use dill instead, to save custom classes as well

New in version 1.1.1: removed Python 2 support.

New in version 1.2.2: automatic swapping of arguments if order is incorrect; correct passing of arguments

New in version 2.0.4: “die” argument for saving as dill

loadobj(filename=None, folder=None, verbose=False, die=None, remapping=None, method='pickle', **kwargs)

Load a file that has been saved as a gzipped pickle file, e.g. by sc.save(). Accepts either a filename (standard usage) or a file object as the first argument. Note that sc.load()/sc.loadobj() are aliases of each other.

Note: be careful when loading pickle files, since a malicious pickle can be used to execute arbitrary code.

When a pickle file is loaded, Python imports any modules that are referenced in it. This is a problem if module has been renamed. In this case, you can use the remapping argument to point to the new modules or classes.

Parameters

• filename (str/Path) – the filename (or full path) to load

• folder (str/Path) – the folder

• verbose (bool) – print details

• die (bool) – whether to raise an exception if errors are encountered (otherwise, load as much as possible)

• remapping (dict) – way of mapping old/unavailable module names to new

• method (str) – method for loading (usually pickle or dill)

• kwargs (dict) – passed to pickle.loads()/dill.loads()

Examples:

obj = sc.load('myfile.obj') # Standard usage
old = sc.load('my-old-file.obj', method='dill', ignore=True) # Load classes from saved files
old = sc.load('my-old-file.obj', remapping={'foo.Bar':cat.Mat}) # If loading a saved object containing a reference to foo.Bar that is now cat.Mat
old = sc.load('my-old-file.obj', remapping={'foo.Bar':('cat', 'Mat')}) # Equivalent to the above

New in version 1.1.0: “remapping” argument

New in version 1.2.2: ability to load non-gzipped pickles; support for dill; arguments passed to loader

saveobj(filename=None, obj=None, compresslevel=5, verbose=0, folder=None, method='pickle', sanitizepath=True, die=False, *args, **kwargs)

Save an object to file as a gzipped pickle – use compression 5 by default, since more is much slower but not much smaller. Once saved, can be loaded with sc.load(). Note that sc.save()/sc.saveobj() are identical.

Parameters

• filename (str/Path) – the filename to save to; if str, passed to sc.makefilepath()

• obj (anything) – the object to save

• compresslevel (int) – the level of gzip compression

• verbose (int) – detail to print

• folder (str) – passed to sc.makefilepath()

• method (str) – whether to use pickle (default) or dill

• die (bool) – whether to fail if the object can’t be pickled (else, try dill)

• sanitizepath (bool) – whether to sanitize the path prior to saving

• args (list) – passed to pickle.dumps()

• kwargs (dict) – passed to pickle.dumps()

Example:

myobj = ['this', 'is', 'a', 'weird', {'object':44}]
sc.save('myfile.obj', myobj)
sc.save('myfile.obj', myobj, method='dill') # Use dill instead, to save custom classes as well

New in version 1.1.1: removed Python 2 support.

New in version 1.2.2: automatic swapping of arguments if order is incorrect; correct passing of arguments

New in version 2.0.4: “die” argument for saving as dill

loadstr(string, verbose=False, die=None, remapping=None)

Like loadobj(), but for a bytes-like string (rarely used).

Example:

obj = sc.objdict(a=1, b=2)
string1 = sc.dumpstr(obj)
string2 = sc.loadstr(string1)
assert string1 == string2

dumpstr(obj=None)

Dump an object as a bytes-like string (rarely used); see sc.loadstr()

rmpath(path=None, *args, die=True, verbose=True, interactive=False, **kwargs)

Remove file(s) and folder(s). Alias to os.remove() (for files) and shutil.rmtree() (for folders).

Parameters

• path (str/Path/list) – file, folder, or list to remove

• args (list) – additional paths to remove

• die (bool) – whether or not to raise an exception if cannot remove

• verbose (bool) – how much detail to print

• interactive (bool) – whether to confirm prior to each deletion

• kwargs (dict) – passed to os.remove()/shutil.rmtree()

Examples:

sc.rmpath('myobj.obj') # Remove a single file
sc.rmpath('myobj1.obj', 'myobj2.obj', 'myobj3.obj') # Remove multiple files
sc.rmpath(['myobj.obj', 'tests']) # Remove a file and a folder interactively
sc.rmpath(sc.getfilelist('tests/*.obj')) # Example of removing multiple files

New in version 2.0.0.

loadtext(filename=None, folder=None, splitlines=False)

Convenience function for reading a text file

Example:

mytext = sc.loadtext('my-document.txt')

savetext(filename=None, string=None, **kwargs)

Convenience function for saving a text file – accepts a string or list of strings; can also save an arbitrary object, in which case it will first convert to a string.

Parameters

• filename (str) – the filename to save to

• string (str) – the string (or object) to save

• kwargs (dict) – passed to np.savetxt() if saving an array

Example:

text = ['Here', 'is', 'a', 'poem']
sc.savetext('my-poem.txt', text)

loadzip(filename=None, outfolder='.', folder=None, extract=True)

Convenience function for reading a zip file

Parameters

• filename (str/path) – the name of the zip file to write to

• outfolder (str/path) – the path location to extract the files to (default: current folder)

• folder (str) – optional additional folder for the filename

• extract (bool) – whether to extract the compressed files; otherwise, load data

Examples:

sc.loadzip('my-files.zip')
data = sc.loadzip('mydata.zip', extract=False)

New in version 2.0.0.

savezip(filename=None, filelist=None, data=None, folder=None, basename=True, sanitizepath=True, verbose=True)

Create a zip file from the supplied list of files (or less commonly, supplied data)

Parameters

• filename (str/path) – the name of the zip file to write to

• filelist (list) – the list of files to compress

• data (dict) – if supplied, instead of files, write this data instead (must be a dictionary of filename keys and data values)

• folder (str) – optional additional folder for the filename

• basename (bool) – whether to use only the file’s basename as the name inside the zip file

• sanitizepath (bool) – whether to sanitize the path prior to saving

• verbose (bool) – whether to print progress

Examples:

scripts = sc.getfilelist('./code/*.py')
sc.savezip('scripts.zip', scripts)

sc.savezip('mydata.zip', data=dict(var1='test', var2=np.random.rand(3)))

New in version 2.0.0: saving data

getfilelist(folder=None, pattern=None, abspath=False, nopath=False, filesonly=False, foldersonly=False, recursive=False, aspath=None)

A shortcut for using glob.

Parameters

• folder (str) – the folder to find files in (default, current)

• pattern (str) – the pattern to match (default, wildcard); can be excluded if part of the folder

• abspath (bool) – whether to return the full path

• nopath (bool) – whether to return no path

• filesonly (bool) – whether to only return files (not folders)

• foldersonly (bool) – whether to only return folders (not files)

• recursive (bool) – passed to glob()

• aspath (bool) – whether to return Path objects

Returns

List of files/folders

Examples:

sc.getfilelist() # return all files and folders in current folder
sc.getfilelist('~/temp', '*.py', abspath=True) # return absolute paths of all Python files in ~/temp folder
sc.getfilelist('~/temp/*.py') # Like above

New in version 1.1.0: “aspath” argument

sanitizefilename(filename, sub='_', allowspaces=False, asciify=True, strict=False, disallowed=None)

Takes a potentially Linux- and Windows-unfriendly candidate file name, and returns a “sanitized” version that is more usable.

Parameters

• filename (str) – the filename to sanitize

• sub (str) – the character to substitute unsafe input characters with

• allowspaces (bool) – whether to allow spaces in the filename

• asciify (bool) – whether to convert the string from Unicode to ASCII

• strict (bool) – whether to remove (almost) all non-alphanumeric characters

• disallowed (str) – optionally supply a custom list of disallowed characters

Example:

bad = 'Nöt*a   file&name?!.doc'
good = sc.sanitizefilename(bad)

New in version 2.0.1: arguments “sub”, “allowspaces”, “asciify”, “strict”, and “disallowed”

makefilepath(filename=None, folder=None, ext=None, default=None, split=False, aspath=None, abspath=True, makedirs=True, checkexists=None, sanitize=False, die=True, verbose=False)

Utility for taking a filename and folder – or not – and generating a valid path from them. By default, this function will combine a filename and folder using os.path.join, create the folder(s) if needed with os.makedirs, and return the absolute path.

Parameters

• filename (str or Path) – the filename, or full file path, to save to – in which case this utility does nothing

• folder (str/Path/list) – the name of the folder to be prepended to the filename; if a list, fed to os.path.join()

• ext (str) – the extension to ensure the file has

• default (str or list) – a name or list of names to use if filename is None

• split (bool) – whether to return the path and filename separately

• aspath (bool) – whether to return a Path object

• abspath (bool) – whether to conver to absolute path

• makedirs (bool) – whether or not to make the folders to save into if they don’t exist

• checkexists (bool) – if False/True, raises an exception if the path does/doesn’t exist

• sanitize (bool) – whether or not to remove special characters from the path; see sc.sanitizefilename() for details

• die (bool) – whether or not to raise an exception if cannot create directory failed (otherwise, return a string)

• verbose (bool) – how much detail to print

Returns

the validated path (or the folder and filename if split=True)

Return type

filepath (str or Path)

Simple example:

filepath = sc.makefilepath('myfile.obj') # Equivalent to os.path.abspath(os.path.expanduser('myfile.obj'))

Complex example:

filepath = makefilepath(filename=None, folder='./congee', ext='prj', default=[project.filename, project.name], split=True, abspath=True, makedirs=True)

Assuming project.filename is None and project.name is “recipe” and ./congee doesn’t exist, this will makes folder ./congee and returns e.g. (‘/home/myname/congee’, ‘recipe.prj’)

New in version 1.1.0: “aspath” argument

path(*args, **kwargs)

Alias to pathlib.Path() with some additional input sanitization:

• None entries are removed

• a list of arguments is converted to separate arguments

New in version 1.2.2.

New in version 2.0.0: handle None or list arguments

PurePath subclass that can make system calls.

Path represents a filesystem path but unlike PurePath, also offers methods to do system calls on path objects. Depending on your system, instantiating a Path will return either a PosixPath or a WindowsPath object. You can also instantiate a PosixPath or WindowsPath directly, but cannot instantiate a WindowsPath on a POSIX system or vice versa.

ispath(obj)

Alias to isinstance(obj, Path).

New in version 2.0.0.

thisdir(file=None, path=None, *args, aspath=None, **kwargs)

Tiny helper function to get the folder for a file, usually the current file. If not supplied, then use the current file.

Parameters

• file (str) – the file to get the directory from; usually __file__

• path (str/list) – additional path to append; passed to os.path.join()

• args (list) – also passed to os.path.join()

• aspath (bool) – whether to return a Path object instead of a string

• kwargs (dict) – passed to Path()

Returns

the full path to the folder (or filename if additional arguments are given)

Return type

filepath (str)

Examples:

thisdir = sc.thisdir() # Get folder of calling file
thisdir = sc.thisdir('.') # Ditto (usually)
thisdir = sc.thisdir(__file__) # Ditto (usually)
file_in_same_dir = sc.thisdir(path='new_file.txt')
file_in_sub_dir = sc.thisdir('..', 'tests', 'mytests.py') # Merge parent folder with sufolders and a file
np_dir = sc.thisdir(np) # Get the folder that Numpy is loaded from (assuming "import numpy as np")

New in version 1.1.0: “as_path” argument renamed “aspath”

New in version 1.2.2: “path” argument

New in version 1.3.0: allow modules

sanitizejson(obj, verbose=True, die=False, tostring=False, **kwargs)

This is the main conversion function for Python data-structures into JSON-compatible data structures (note: sc.sanitizejson()/sc.jsonify() are identical).

Parameters

• obj (any) – almost any kind of data structure that is a combination of list, numpy.ndarray, odicts, etc.

• verbose (bool) – level of detail to print

• die (bool) – whether or not to raise an exception if conversion failed (otherwise, return a string)

• tostring (bool) – whether to return a string representation of the sanitized object instead of the object itself

• kwargs (dict) – passed to json.dumps() if tostring=True

Returns

the converted object that should be JSON compatible, or its representation as a string if tostring=True

Return type

object (any or str)

Version: 2020apr11

jsonify(obj, verbose=True, die=False, tostring=False, **kwargs)

This is the main conversion function for Python data-structures into JSON-compatible data structures (note: sc.sanitizejson()/sc.jsonify() are identical).

Parameters

• obj (any) – almost any kind of data structure that is a combination of list, numpy.ndarray, odicts, etc.

• verbose (bool) – level of detail to print

• die (bool) – whether or not to raise an exception if conversion failed (otherwise, return a string)

• tostring (bool) – whether to return a string representation of the sanitized object instead of the object itself

• kwargs (dict) – passed to json.dumps() if tostring=True

Returns

the converted object that should be JSON compatible, or its representation as a string if tostring=True

Return type

object (any or str)

Version: 2020apr11

loadjson(filename=None, folder=None, string=None, fromfile=True, **kwargs)

Convenience function for reading a JSON file (or string).

Parameters

• filename (str) – the file to load, or the JSON object if using positional arguments

• folder (str) – folder if not part of the filename

• string (str) – if not loading from a file, a string representation of the JSON

• fromfile (bool) – whether or not to load from file

• kwargs (dict) – passed to json.load()

Returns

the JSON object

Return type

output (dict)

Examples:

json = sc.loadjson('my-file.json')
json = sc.loadjson(string='{"a":null, "b":[1,2,3]}')

savejson(filename=None, obj=None, folder=None, die=True, indent=2, keepnone=False, sanitizepath=True, **kwargs)

Convenience function for saving to a JSON file.

Parameters

• filename (str) – the file to save

• obj (anything) – the object to save; if not already in JSON format, conversion will be attempted

• folder (str) – folder if not part of the filename

• die (bool) – whether or not to raise an exception if saving an empty object

• indent (int) – indentation to use for saved JSON

• keepnone (bool) – allow sc.savejson(None) to return ‘null’ rather than raising an exception

• sanitizepath (bool) – whether to sanitize the path prior to saving

• kwargs (dict) – passed to json.dump()

Returns

The filename saved to

Example:

json = {'foo':'bar', 'data':[1,2,3]}
sc.savejson('my-file.json', json)

loadyaml(filename=None, folder=None, string=None, fromfile=True, safe=False, loader=None)

Convenience function for reading a YAML file (or string).

Parameters

• filename (str) – the file to load, or the YAML object if using positional arguments

• folder (str) – folder if not part of the filename

• string (str) – if not loading from a file, a string representation of the YAML

• fromfile (bool) – whether or not to load from file

• safe (bool) – whether to use the safe loader

• loader (Loader) – custom YAML loader (takes precedence over safe)

Returns

the YAML object

Return type

output (dict)

Examples:

yaml = sc.loadyaml('my-file.yaml')
yaml = sc.loadyaml(string='{"a":null, "b":[1,2,3]}')

saveyaml(filename=None, obj=None, folder=None, die=True, keepnone=False, dumpall=False, sanitizepath=True, **kwargs)

Convenience function for saving to a YAML file.

Parameters

• filename (str) – the file to save (if empty, return string representation of the YAML instead)

• obj (anything) – the object to save

• folder (str) – folder if not part of the filename

• die (bool) – whether or not to raise an exception if saving an empty object

• indent (int) – indentation to use for saved YAML

• keepnone (bool) – allow sc.saveyaml(None) to return ‘null’ rather than raising an exception

• dumpall (bool) – if True, treat a list input as separate YAML pages

• sanitizepath (bool) – whether to sanitize the path prior to saving

• kwargs (dict) – passed to yaml.dump()

Returns

The filename saved to

Examples:

yaml = {'foo':'bar', 'data':[1,2,3]}
sc.saveyaml('my-file.yaml', yaml) # Save to file

string = sc.saveyaml(obj=yaml) # Export to string

jsonpickle(obj, tostring=False)

Save any Python object to a JSON file using jsonpickle.

Parameters

• obj (any) – the object to pickle as a JSON

• tostring (bool) – whether to return a string (rather than the JSONified Python object)

Returns

Either a string or a Python object for the JSON

Wrapper for the jsonpickle library: https://jsonpickle.github.io/

jsonunpickle(json)

Use jsonpickle to restore an object (see jsonpickle())

class Blobject(source=None, name=None, filename=None, blob=None)

A wrapper for a binary file – rarely used directly.

So named because it’s an object representing a blob.

“source” is a specification of where to get the data from. It can be anything supported by Blobject.load() which are (a) a filename, which will get loaded, or (b) a io.BytesIO which will get dumped into this instance

Alternatively, can specify blob which is a binary string that gets stored directly in the blob attribute

load(source=None)

This function loads the spreadsheet from a file or object. If no input argument is supplied, then it will read self.bytes, assuming it exists.

save(filename=None)

This function writes the spreadsheet to a file on disk.

tofile(output=True)

Return a file-like object with the contents of the file.

This can then be used to open the workbook from memory without writing anything to disk e.g.

• book = openpyxl.load_workbook(self.tofile())

• book = xlrd.open_workbook(file_contents=self.tofile().read())

freshbytes()

Refresh the bytes object to accept new data

class Spreadsheet(*args, **kwargs)

A class for reading and writing Excel files in binary format. No disk IO needs to happen to manipulate the spreadsheets with openpyxl (or xlrd or pandas).

New in version 1.3.0: Changed default from xlrd to openpyxl and added self.wb attribute to avoid the need to reload workbooks.

Examples:

.. py:method:: Spreadsheet.new(**kwargs)

module

sciris.sc_fileio

Shortcut method to create a new openpyxl workbook

xlrd(reload=False, store=True, **kwargs)

Legacy method to load from xlrd

openpyxl(reload=False, store=True, **kwargs)

Return a book as opened by openpyxl

openpyexcel(*args, **kwargs)

Legacy name for openpyxl()

pandas(reload=False, store=True, **kwargs)

Return a book as opened by pandas

update(book)

Updated the stored spreadsheet with book instead

readcells(wbargs=None, *args, **kwargs)

Alias to loadspreadsheet()

writecells(cells=None, startrow=None, startcol=None, vals=None, sheetname=None, sheetnum=None, verbose=False, wbargs=None)

Specify cells to write. Can supply either a list of cells of the same length as the values, or else specify a starting row and column and write the values from there.

Examples:

S = sc.Spreadsheet()
S.writecells(cells=['A6','B7'], vals=['Cat','Dog']) # Method 1
S.writecells(cells=[np.array([2,3])+i for i in range(2)], vals=['Foo', 'Bar']) # Method 2
S.writecells(startrow=14, startcol=1, vals=np.random.rand(3,3)) # Method 3
S.save('myfile.xlsx')

loadspreadsheet(filename=None, folder=None, fileobj=None, sheet=0, header=1, asdataframe=None, method='pandas', **kwargs)

Load a spreadsheet as a dataframe or a list of lists.

By default, an alias to pandas.read_excel() with a header, but also supports loading via openpyxl or xlrd. Read from either a filename or a file object.

Parameters

• filename (str) – filename or path to read

• folder (str) – optional folder to use with the filename

• fileobj (obj) – load from file object rather than path

• sheet (str/int/list) – name or number of sheet(s) to use (default 0)

• asdataframe (bool) – whether to return as a pandas/Sciris dataframe (default True)

• header (bool) – whether the 0-th row is to be read as the header

• method (str) – how to read (default ‘pandas’, other choices ‘openpyxl’ and ‘xlrd’)

• kwargs (dict) – passed to pd.read_excel(), openpyxl(), etc.

Examples:

df = sc.loadspreadsheet('myfile.xlsx') # Alias to pd.read_excel(header=1)
wb = sc.loadspreadsheet('myfile.xlsx', method='openpyxl') # Returns workbook
data = sc.loadspreadsheet('myfile.xlsx', method='xlrd', asdataframe=False) # Returns raw data; requires xlrd

New in version 1.3.0: change default from xlrd to pandas; renamed sheetname and sheetnum arguments to sheet.

savespreadsheet(filename=None, data=None, folder=None, sheetnames=None, close=True, workbook_args=None, formats=None, formatdata=None, verbose=False)

Semi-simple function to save data nicely to Excel.

Parameters

• filename (str) – Excel file to save to

• data (list/array) – data to write to the spreadsheet

• folder (str) – if supplied, merge with the filename to make a path

• sheetnames (list) – if data is supplied as a list of arrays, save each entry to a different sheet

• close (bool) – whether to close the workbook after saving

• workbook_args (dict) – arguments passed to xlxwriter.Workbook()

• formats (dict) – a definition of different types of formatting (see examples below)

• formatdata (array) – an array of which formats go where

• verbose (bool) – whether to print progress

Examples:

import sciris as sc
import pylab as pl

# Simple example
testdata1 = np.random.rand(8,3)
sc.savespreadsheet(filename='test1.xlsx', data=testdata1)

# Include column headers
test2headers = [['A','B','C']] # Need double brackets to get right shape
test2values = np.random.rand(8,3).tolist()
testdata2 = test2headers + test2values
sc.savespreadsheet(filename='test2.xlsx', data=testdata2)

# Multiple sheets
testdata3 = [np.random.rand(10,10), np.random.rand(20,5)]
sheetnames = ['Ten by ten', 'Twenty by five']
sc.savespreadsheet(filename='test3.xlsx', data=testdata3, sheetnames=sheetnames)

# Supply data as an odict
testdata4 = sc.odict([('First sheet', np.random.rand(6,2)), ('Second sheet', np.random.rand(3,3))])
sc.savespreadsheet(filename='test4.xlsx', data=testdata4)

# Include formatting
nrows = 15
ncols = 3
formats = {
    'header':{'bold':True, 'bg_color':'#3c7d3e', 'color':'#ffffff'},
    'plain': {},
    'big':   {'bg_color':'#ffcccc'}
}
testdata5  = np.zeros((nrows+1, ncols), dtype=object) # Includes header row
formatdata = np.zeros((nrows+1, ncols), dtype=object) # Format data needs to be the same size
testdata5[0,:] = ['A', 'B', 'C'] # Create header
testdata5[1:,:] = np.random.rand(nrows,ncols) # Create data
formatdata[1:,:] = 'plain' # Format data
formatdata[testdata5>0.7] = 'big' # Find "big" numbers and format them differently
formatdata[0,:] = 'header' # Format header
sc.savespreadsheet(filename='test5.xlsx', data=testdata5, formats=formats, formatdata=formatdata)

New in version 2.0.0: allow arguments to be passed to the Workbook.

class Failed(*args, **kwargs)

An empty class to represent a failed object loading

class Empty(*args, **kwargs)

Another empty class to represent a failed object loading, but do not proceed with setstate