All notable changes to this project will be documented in this file.
By import convention, components of the Sciris library are listed beginning with
Version 1.2.3 (2021-08-27)¶
Fixed a bug with
verbose > 1. (Thanks to Nick Scott and Romesh Abeysuriya.)
sc.rolling()as a shortcut to pandas’ rolling average function.
sc.findlast(), to allow returning no indices without error.
Version 1.2.2 (2021-08-21)¶
New functions and methods¶
A new class,
sc.autolist(), is available to simplify appending to lists, e.g.
ls = sc.autolist(); ls += 'not a list'.
sc.freeze()as a programmatic equivalent of
sc.require()as a flexible way of checking (or asserting) environment requirements, e.g.
sc.path()as an alias to
Added an even more robust unpickler, that should be able to recover data even if exceptions are raised when unpickling.
sc.loadobj()to allow loading standard (not gzipped) pickles and from
sc.saveobj()to automatically swap arguments if the object is supplied first, then the filename.
sc.asd()to allow more flexible argument passing to the optimized function; also updated
verboseto allow skipping iterations.
sc.thisdir()to more easily allow subfolders/files.
Instead of being separate function definitions,
sc.jsonify()are now identical to their aliases (e.g.
sc.dateformatter()now allows a
rotationargument, since date labels often collide.
sc.date()can now read additional numeric dates, e.g.
sc.promotetolist()now converts (rather than wraps) ranges and dict_keys objects to lists. To restore the previous behavior, use the argument
start_dayargument has been renamed
sc.date()has been renamed
outformat, to differentiate from
Version 1.2.1 (2021-07-07)¶
openpyxlas a Sciris dependency, since it was removed from pandas.
sc.datedelta(), a function that wraps
datetime.timedeltato easily do date operations on strings, e.g.
Added additional supported date formats to
sc.readdate(), along with new
dateformat, to read common day-month-year and month-day-year formats.
Added the ability for
Errors loading pickles from
sc.load()are now more informative.
Version 1.2.0 (2021-07-05)¶
New functions and methods¶
sc.figlayout()as an alias to both
sc.midpointnorm()as an alias to Matplotlib’s
TwoSlopeNorm; it can also be used in e.g.
sc.dateformatter(), which will (semi-)automatically format the x-axis using dates.
sc.ismac(). These are all shortcuts for checking
sc.cpu_count()as a simple alias for
sc.checkmem()from failing when an attribute was
Fixed a file handle that was being left open by
sc.odictand derived classes; adding two dictionaries is the same as calling
Updated nested dictionary functions, and added them as methods to
sc.odict()and derived classes (like
sc.objdict()); for example, you can now do
nestedobj = sc.objdict(); nestedobj.setnested(['a','b','c'], 4).
sc.odict.enumvalues()as an alias to
sc.commaticks()to use better formatting.
sc.SIticks(); now, the first argument can be an
Figureobject, or a list of axes.
sc.get_rows_cols()to optionally create subplots, rather than just return the number of rows/columns.
sc.heading()to handle arguments the same way as
sc.heading([1,2,3], 'is a list').
Allowed more flexibility with the
sc.parallelize(): it can now be a fraction, representing a fraction of available CPUs. Also, it will now never exceed the number of tasks to be run.
sc.suggest()to modify the threshold to be based on the length of the input word.
Version 1.1.1 (2021-03-17)¶
The implementations of
sc.objdict()have been updated, to allow for more flexible use of the
defaultdictargument, including better nesting and subclassing.
serialargument has been added to
sc.parallelize()to allow for quick debugging.
Legacy support for Python 2 has been removed from
A fallback method for
gitpython) has been added, in case reading from the filesystem fails.
Version 1.1.0 (2021-03-12)¶
sc.mergelists()is similar to
sc.mergedicts(): it will take a sequence of inputs and attempt to merge them into a list.
sc.transposelist()will perform a transposition on a list of lists: for example, a list of 10 lists (or tuples) each of length 3 will be transformed into a list of 3 lists each of length 10.
sc.newlinejoin()are shortcuts to
'\n'.join(items), respectively. The latter is especially useful inside f-strings since you cannot use the
sc.day()now returns a numeric array when an array of datetime objects is passed to it; a bug which was introduced in version 1.0.2 which meant it returned an object array instead.
Slices with numeric start and stop indices have been fixed for
sc.objatt()now correctly handles objects with slots instead of a dict.
sc.loadobj()now accepts a
remappingargument, which lets the user load old pickle files even if the modules no longer exist.
Most file functions (e.g.
sc.getfilelist()now accept an
aspathargument, which, if
True, will return a
pathlib.Pathobject instead of a string.
Most array-returning functions, such as
sc.cat(), now accept a
copyargument and other keywords; these keywords are passed to
np.array(), allowing e.g. the
dtypeto be set.
A fallback option for
sc.findinds()has been implemented, allowing it to work even if the input array isn’t numeric.
sc.odict()now has a
defaultdictargument, which lets you use it like a defaultdict as well as an ordered dict.
transposeargument for methods like
enumvalues(), which will return a tuple of lists instead of a list of tuples.
sc.objdict()now prints out differently, to distinguish it from an
sc.promotetolist()has a new
coerceargument, which will convert that data type into a list (instead of wrapping it).
sc.toarray()have been added as aliases of
sc.promotetoarray(), respectively. You may use whichever you prefer.
skipnonekeyword has been removed from
sc.promotetoarray()and replaced with
keepnone(which does something slightly different).
Exceptions have been made more specific (e.g.
Test code coverage has been increased significantly (from 63% to 84%).
Version 1.0.2 (2021-03-10)¶
Fixed bug (introduced in version 1.0.1) with
sc.readdate()returning only the first element of a list of a dates.
Fixed bug (introduced in version 1.0.1) with
sc.date()treating an integer as a timestamp rather than an integer number of days when a start day is supplied.
sc.day()to always return consistent output types (e.g. if an array is supplied as an input, an array is supplied as an output).
Version 1.0.1 (2021-03-01)¶
Fixed bug with Matplotlib 3.4.0 also defining colormap
'turbo', which caused Sciris to fail to load.
Added a new function,
sc.orderlegend(), that lets you specify the order you want the legend items to appear.
Fixed bug with paths returned by
Fixed bug with
sc.loadjson()only reading from a string if
Fixed recursion issue with printing
sc.approx()to be an alias to
np.isclose(); this function may be removed in future versions.
np.isclose(), allowing for greater flexibility.
sc.objdict()to differ from
sc.maximize()to work on more platforms (but still not inline or on Macs).
Improved the flexiblity of
sc.htmlify()to handle tabs and other kinds of newlines.
Added additional checks to
sc.prepr()to avoid failing on recursive objects.
sc.mergedicts()to return the same type as the first dict supplied.
sc.date()to support timestamps as well as strings.
sc.gitinfo()to try each piece independently, so if it fails on one (e.g., extracting the date) it will still return the other pieces (e.g., the hash).
xlrdto 1.2.0 since later versions fail to read xlsx files.
Version 1.0.0 (2020-11-30)¶
This major update (and official release!) includes many new utilities adopted from the Covasim and Atomica libraries, as well as important improvements and bugfixes for parallel processing, object representation, and file I/O.
sc.findlast()return the first and last indices, respectively, of what
sc.findinds()would return. These keywords (
last) can also be passed directly to
sc.randround()probabilistically rounds numbers to the nearest integer; e.g. 1.2 will round down 80% of the time.
sc.cat()is a generalization of
np.concatenate()that handles arbitrary types and numbers of inputs.
sc.isarray()checks if the object is a Numpy array.
A new diverging colormap,
'orangeblue', has been added (courtesy Prashanth Selvaraj). It is rather pretty; you should try it out.
sc.get_rows_cols()solves the small but annoying issue of trying to figure out how many rows and columns you need to plot N axes. It is similar to
np.unravel_index(), but allows the desired aspect ratio to be varied.
sc.maximize()maximizes the current figure window.
sc.date()will convert practically anything to a date.
sc.day()will convert practically anything to an integer number of days from a starting point; for example,
sc.day(sc.now())returns the number of days since Jan. 1st.
sc.daydiff()computes the number of days between two or more start and end dates.
sc.daterange()returns a list of date strings or date objects between the start and end dates.
sc.datetoyear()converts a date to a decimal year (from Romesh Abeysuriya via Atomica).
The “flagship” functions
sc.saveobj()now have shorter aliases:
sc.save(). These functions can be used interchangeably.
A convenience function,
sc.toctic(), has been added that does
sc.toc(); sc.tic(), i.e. for sequentially timing multiple blocks of code.
sc.checkram()reports the current process’ RAM usage at the current moment in time; useful for debugging memory leaks.
sc.getcaller()returns the name and line number of the calling function; useful for logging and version control purposes.
sc.nestedloop()iterates over lists in the specified order (from Romesh Abeysuriya via Atomica).
sc.parallel_progress()runs a function in parallel whilst displaying a single progress bar across all processes (from Romesh Abeysuriya via Atomica).
An experimental function,
sc.asobj(), has been added that lets any dictionary-like object be used with attributes instead (i.e.
Bugfixes and other improvements¶
sc.parallelize()now uses the
multiprocesslibrary instead of
multiprocessing. This update fixes bugs with trying to run parallel processing in certain environments (e.g., in Jupyter notebooks). This function also returns a more helpful error message when running in the wrong context on Windows.
sc.prepr()has been updated to use a simpler method of parsing objects for display; this should be faster and more robust. A default 3 second time limit has also been added.
sc.savejson()now uses an indent of 2 by default, leading to much more human-readable JSON files.
sc.gitinfo()has been updated to use the code from Atomica’s
fast_gitinfo()instead (courtesy Romesh Abeysuriya).
sc.thisdir()now no longer requires the
__file__argument to be supplied to get the current folder.
sc.readdate()can now handle a list of dates.
sc.getfilelist()now has more options, such as to return the absolute path or no path, as well as handling file matching patterns more flexibly.
sc.Empty, which may be encountered when loading a corrupted pickle file, are now exposed to the user (before they could only be accessed via
sc.perturb()can now use either uniform or normal perturbations via the
sc.quantile()has been removed. Please use
np.quantile()instead (though admittedly, it is extremely unlikely you were using it to begin with).
sc.scaleratio()has been renamed
sc.normsum(), since it normalizes an array by the sum.
Module imports were moved to inside functions, improving Sciris loading time by roughly 30%.
All tests were refactored to be in consistent format, increasing test coverage by roughly 50%.
Continuous integration testing was updated to use GitHub Actions instead of Travis/Tox.
Version 0.17.4 (2020-08-11)¶
sc.mprofile()now return the line profiler instance for later use (e.g., to extract additional statistics).
sc.prepr()(also used in
sc.prettyobj()) can now support objects with slots instead of dicts.
Version 0.17.3 (2020-07-21)¶
sc.parallelize()now explicitly deep-copies objects, since on some platforms this copying does not take place as part of the parallelization process.
Version 0.17.2 (2020-07-13)¶
sc.search()is a new function to find nested attributes/keys within objects or dictionaries.
Version 0.17.1 (2020-07-07)¶
sc.Blobjecthas been modified to allow more flexibility with saving (e.g.,
Version 0.17.0 (2020-04-27)¶
sc.mprofile()has been added, which does memory profiling just like
sc.progressbar()has been added, which prints a progress bar.
sc.jsonunpickle()have been added, wrapping the module of the same name, to convert arbitrary objects to JSON.
sc.jsonify()checks objects for a
to_json()method, handling e.g Pandas dataframes, and falls back to
sc.jsonpickle()instead of raising an exception for unknown object types.
python-levenshteinfor fuzzy string matching.
sc.saveobj()now uses protocol 4 instead of the latest by default, to avoid backwards incompatibility issues caused by using protocol 5 (only compatible with Python 3.8).
sc.odict()and related classes now raise
sc.KeyNotFoundErrorexceptions. These are derived from
KeyError, but fix a bug in the string representation to allow multi-line error messages.
Rewrote all tests to be pytest-compatible.
Version 0.16.8 (2020-04-11)¶
sc.makefilepath()now has a
checkexistsflag, which will optionally raise an exception if the file does (or doesn’t) exist.
sc.fast_uuid()now work with non-integer inputs, e.g.,
sc.thisdir()now accepts additional arguments, so can be used to form a full path, e.g.
sc.checkmem()has better parsing of objects.
sc.prepr()now lists properties of objects, and has some aesthetic improvements.