sciris.sc_utils

Miscellaneous utilities for type checking, printing, dates and times, etc.

Note: there are a lot! The design philosophy has been that it’s easier to ignore a function that you don’t need than write one from scratch that you do need.

Highlights:
  • dcp(): shortcut to copy.deepcopy()

  • pp(): shortcut to pprint.pprint()

  • isnumber(): checks if something is any number type

  • tolist(): converts any object to a list, for easy iteration

  • toarray(): tries to convert any object to an array, for easy use with numpy

  • mergedicts(): merges any set of inputs into a dictionary

  • mergelists(): merges any set of inputs into a list

  • runcommand(): simple way of executing a shell command

  • download(): download multiple URLs in parallel

Functions

asciify

Convert an arbitrary Unicode string to ASCII.

checktype

A convenience function for checking instances.

compareversions

Function to compare versions, expecting both arguments to be a string of the format 1.2.3, but numeric works too.

cp

Shortcut to perform a shallow copy operation

dcp

Shortcut to perform a deep copy operation

download

Download one or more URLs in parallel and return output or save them to disk.

fast_uuid

Create a fast UID or set of UIDs.

flexstr

Try converting any object to a "regular" string (i.e.

freeze

Alias for pip freeze.

getcaller

Try to get information on the calling function, but fail gracefully.

getplatform

Return the name of the current platform: 'linux', 'windows', 'mac', or 'other'.

gitinfo

Retrieve git info

htmlify

Convert a string to its HTML representation by converting unicode characters, characters that need to be escaped, and newlines.

importbyname

Import modules by name.

isarray

Check whether something is a Numpy array, and optionally check the dtype.

isiterable

Simply determine whether or not the input is iterable.

islinux

Alias to sc.getplatform('linux')

ismac

Alias to sc.getplatform('mac')

isnumber

Determine whether or not the input is a number.

isstring

Determine whether or not the input is string-like (i.e., str or bytes).

iswindows

Alias to sc.getplatform('windows')

mergedicts

Small function to merge multiple dicts together.

mergelists

Merge multiple lists together.

newlinejoin

Alias to strjoin(*args, sep='\n').

pp

Shortcut for pretty-printing the object.

promotetoarray

Small function to ensure consistent format for things that should be arrays (note: toarray() and promotetoarray() are identical).

promotetolist

Make sure object is always a list (note: tolist()/promotetolist() are identical).

require

Check whether environment requirements are met.

runcommand

Make it easier to run shell commands.

sha

Shortcut for the standard hashing (SHA) method

strjoin

Like string join(), but handles more flexible inputs, converts items to strings.

strsplit

Convenience function to split common types of strings.

suggest

Return suggested item

swapdict

Swap the keys and values of a dictionary.

toarray

Small function to ensure consistent format for things that should be arrays (note: toarray() and promotetoarray() are identical).

tolist

Make sure object is always a list (note: tolist()/promotetolist() are identical).

traceback

Shortcut for accessing the traceback

transposelist

Convert e.g.

uniquename

Given a name and a list of other names, find a replacement to the name that doesn't conflict with the other names, and pass it back.

urlopen

Download a single URL.

uuid

Shortcut for creating a UUID; default is to create a UUID4.

wget

Download a single URL.

Classes

LazyModule

Create a "lazy" module that is loaded if and only if an attribute is called.

Link

A class to differentiate between an object and a link to an object.

autolist

A simple extension to a list that defines add methods to simplify appending and extension.

prettyobj

Use pretty repr for objects, instead of just showing the type and memory pointer (the Python default for objects).

Exceptions

KeyNotFoundError

A tiny class to fix repr for KeyErrors.

LinkException

An exception to raise when links are broken, for exclusive use with the Link class.

fast_uuid(which=None, length=None, n=1, secure=False, forcelist=False, safety=1000, recursion=0, recursion_limit=10, verbose=True)[source]

Create a fast UID or set of UIDs. Note: for certain applications, sc.uuid() is faster than sc.fast_uuid()!

Parameters
  • which (str) – the set of characters to choose from (default ascii)

  • length (int) – length of UID (default 6)

  • n (int) – number of UIDs to generate

  • secure (bool) – whether to generate random numbers from sources provided by the operating system

  • forcelist (bool) – whether or not to return a list even for a single UID (used for recursive calls)

  • safety (float) – ensure that the space of possible UIDs is at least this much larger than the number requested

  • recursion (int) – the recursion level of the call (since the function calls itself if not all UIDs are unique)

  • recursion_limit (int) – Maximum number of times to try regeneraring keys

  • verbose (bool) – whether to show progress

Returns

a string UID, or a list of string UIDs

Return type

uid (str or list)

Example:

uuids = sc.fast_uuid(n=100) # Generate 100 UUIDs

Inspired by https://stackoverflow.com/questions/2257441/random-string-generation-with-upper-case-letters-and-digits/30038250#30038250

uuid(uid=None, which=None, die=False, tostring=False, length=None, n=1, **kwargs)[source]

Shortcut for creating a UUID; default is to create a UUID4. Can also convert a UUID.

Parameters
  • uid (str or uuid) – if a string, convert to an actual UUID; otherwise, return unchanged

  • which (int or str) – if int, choose a Python UUID function; otherwise, generate a random alphanumeric string (default 4)

  • die (bool) – whether to fail for converting a supplied uuid (default False)

  • tostring (bool) – whether or not to return a string instead of a UUID object (default False)

  • length (int) – number of characters to trim to, if returning a string

  • n (int) – number of UUIDs to generate; if n>1, return a list

Returns

the UID object

Return type

uid (UUID or str)

Examples:

sc.uuid() # Alias to uuid.uuid4()
sc.uuid(which='hex') # Creates a length-6 hex string
sc.uuid(which='ascii', length=10, n=50) # Creates 50 UUIDs of length 10 each using the full ASCII character set
dcp(obj, die=True, verbose=True)[source]

Shortcut to perform a deep copy operation

Almost identical to copy.deepcopy(), but optionally fall back to copy() if deepcopy fails.

Parameters
  • die (bool) – if False, fall back to copy()

  • verbose (bool) – if die is False, then print a warning if deepcopy() fails

New in version 2.0.0: default die=True instead of False

cp(obj, die=True, verbose=True)[source]

Shortcut to perform a shallow copy operation

Almost identical to copy.copy(), but optionally allow failures

New in version 2.0.0: default die=True instead of False

pp(obj, jsonify=True, doprint=None, output=False, verbose=False, **kwargs)[source]

Shortcut for pretty-printing the object.

Almost identical to pprint.pprint(), but can also be used as an alias for pprint.pformat().

Parameters
  • obj (any) – object to print

  • jsonify (bool) – whether to first convert the object to JSON, to handle things like ordered dicts nicely

  • doprint (bool) – whether to show output (default true)

  • output (bool) – whether to return output as a string (default false)

  • verbose (bool) – whether to show warnings when jsonifying the object

  • kwargs (dict) – passed to pprint.pprint()

New in version 1.3.1: output argument

sha(obj, encoding='utf-8', digest=False)[source]

Shortcut for the standard hashing (SHA) method

Equivalent to hashlib.sha224().

Parameters
  • obj (any) – the object to be hashed; if not a string, converted to one

  • encoding (str) – the encoding to use

  • digest (bool) – whether to return the hex digest instead of the hash objet

Example:

sha1 = sc.sha(dict(foo=1, bar=2), digest=True)
sha2 = sc.sha(dict(foo=1, bar=2), digest=True)
sha3 = sc.sha(dict(foo=1, bar=3), digest=True)
assert sha1 == sha2
assert sha2 != sha3
freeze(lower=False)[source]

Alias for pip freeze.

Parameters

lower (bool) – convert all keys to lowercase

Example:

assert 'numpy' in sc.freeze() # One way to check for versions

New in version 1.2.2.

require(reqs=None, *args, exact=False, detailed=False, die=True, verbose=True, **kwargs)[source]

Check whether environment requirements are met. Alias to pkg_resources.require().

Parameters
  • reqs (list/dict) – a list of strings, or a dict of package names and versions

  • args (list) – additional requirements

  • kwargs (dict) – additional requirements

  • exact (bool) – use ‘==’ instead of ‘>=’ as the default comparison operator if not specified

  • detailed (bool) – return a dict of which requirements are/aren’t met

  • die (bool) – whether to raise an exception if requirements aren’t met

  • verbose (bool) – print out the exception if it’s not being raised

Examples:

sc.require('numpy')
sc.require(numpy='')
sc.require(reqs={'numpy':'1.19.1', 'matplotlib':'3.2.2'})
sc.require('numpy>=1.19.1', 'matplotlib==3.2.2', die=False)
sc.require(numpy='1.19.1', matplotlib='==4.2.2', die=False, detailed=True)

New in version 1.2.2.

traceback(*args, **kwargs)[source]

Shortcut for accessing the traceback

Alias for traceback.format_exc().

getplatform(expected=None, die=False)[source]

Return the name of the current platform: ‘linux’, ‘windows’, ‘mac’, or ‘other’. Alias (kind of) to sys.platform.

Parameters
  • expected (str) – if not None, check if the current platform is this

  • die (bool) – if True and expected is defined, raise an exception

Example:

sc.getplatform() # Get current name of platform
sc.getplatform('windows', die=True) # Raise an exception if not on Windows
iswindows(die=False)[source]

Alias to sc.getplatform('windows')

islinux(die=False)[source]

Alias to sc.getplatform('linux')

ismac(die=False)[source]

Alias to sc.getplatform('mac')

asciify(string, form='NFKD', encoding='ascii', errors='ignore', **kwargs)[source]

Convert an arbitrary Unicode string to ASCII.

Parameters
  • form (str) – the type of Unicode normalization to use

  • encoding (str) – the output string to encode to

  • errors (str) – how to handle errors

  • kwargs (dict) – passed to string.decode()

Example:

sc.asciify(‘föö→λ ∈ ℝ’) # Returns ‘foo R’

New in version 2.0.1.

urlopen(url, filename=None, save=False, headers=None, params=None, data=None, prefix='http', convert=True, die=False, return_response=False, verbose=False)[source]

Download a single URL.

Alias to urllib.request.urlopen(url).read(). See also sc.download() for downloading multiple URLs. Note: sc.urlopen()/sc.wget() are aliases.

Parameters
  • url (str) – the URL to open, either as GET or POST

  • filename (str) – if supplied, save to file instead of returning output

  • save (bool) – if supplied instead of filename, then use the default filename

  • headers (dict) – a dictionary of headers to pass

  • params (dict) – a dictionary of parameters to pass to the GET request

  • data (dict) –

  • prefix (str) – the string to ensure the URL starts with (else, add it)

  • convert (bool) – whether to convert from bytes to string

  • die (bool) – whether to raise an exception if converting to text failed

  • return_response (bool) – whether to return the response object instead of the output

  • verbose (bool) – whether to print progress

Examples:

html = sc.urlopen('sciris.org') # Retrieve into variable html
sc.urlopen('http://sciris.org', filename='sciris.html') # Save to file sciris.html
sc.urlopen('http://sciris.org', save=True, headers={'User-Agent':'Custom agent'}) # Save to the default filename (here, sciris.org), with headers
New in version 2.0.0: renamed from wget to urlopen; new arguments
New in version 2.0.1: creates folders by default if they do not exist
New in version 2.0.4: “prefix” argument, e.g. prepend “http://” if not present
wget(url, filename=None, save=False, headers=None, params=None, data=None, prefix='http', convert=True, die=False, return_response=False, verbose=False)

Download a single URL.

Alias to urllib.request.urlopen(url).read(). See also sc.download() for downloading multiple URLs. Note: sc.urlopen()/sc.wget() are aliases.

Parameters
  • url (str) – the URL to open, either as GET or POST

  • filename (str) – if supplied, save to file instead of returning output

  • save (bool) – if supplied instead of filename, then use the default filename

  • headers (dict) – a dictionary of headers to pass

  • params (dict) – a dictionary of parameters to pass to the GET request

  • data (dict) –

  • prefix (str) – the string to ensure the URL starts with (else, add it)

  • convert (bool) – whether to convert from bytes to string

  • die (bool) – whether to raise an exception if converting to text failed

  • return_response (bool) – whether to return the response object instead of the output

  • verbose (bool) – whether to print progress

Examples:

html = sc.urlopen('sciris.org') # Retrieve into variable html
sc.urlopen('http://sciris.org', filename='sciris.html') # Save to file sciris.html
sc.urlopen('http://sciris.org', save=True, headers={'User-Agent':'Custom agent'}) # Save to the default filename (here, sciris.org), with headers
New in version 2.0.0: renamed from wget to urlopen; new arguments
New in version 2.0.1: creates folders by default if they do not exist
New in version 2.0.4: “prefix” argument, e.g. prepend “http://” if not present
download(url, *args, filename=None, save=True, parallel=True, verbose=True, **kwargs)[source]

Download one or more URLs in parallel and return output or save them to disk.

A wrapper for sc.urlopen(), except with save=True by default.

Parameters
  • url (str/list/dict) – either a single URL, a list of URLs, or a dict of URL:filename pairs

  • *args (list) – additional URLs to download

  • filename (str/list) – either a string or a list of the same length as url (if not supplied, return output)

  • save (bool) – if supplied instead of filename, then use the default filename

  • parallel (bool) – whether to download multiple URLs in parallel

  • verbose (bool) – whether to print progress (if verbose=2, print extra detail on each downloaded URL)

  • **kwargs (dict) – passed to sc.urlopen()

Examples:

html = sc.download('http://sciris.org') # Download a single URL
data = sc.download('http://sciris.org', 'http://covasim.org') # Download two in parallel
sc.download({'http://sciris.org':'sciris.html', 'http://covasim.org':'covasim.html'}) # Downlaod two and save to disk
sc.download(['http://sciris.org', 'http://covasim.org'], filename=['sciris.html', 'covasim.html']) # Ditto

New in version 2.0.0.

htmlify(string, reverse=False, tostring=False)[source]

Convert a string to its HTML representation by converting unicode characters, characters that need to be escaped, and newlines. If reverse=True, will convert HTML to string. If tostring=True, will convert the bytestring back to Unicode.

Examples:

output = sc.htmlify('foo&\nbar') # Returns b'foo&amp;<br>bar'
output = sc.htmlify('föö&\nbar', tostring=True) # Returns 'f&#246;&#246;&amp;&nbsp;&nbsp;&nbsp;&nbsp;bar'
output = sc.htmlify('foo&amp;<br>bar', reverse=True) # Returns 'foo&\nbar'
flexstr(arg, force=True)[source]

Try converting any object to a “regular” string (i.e. str), but proceed if it fails. Note: this function calls repr() rather than str() to ensure a more robust representation of objects.

isiterable(obj)[source]

Simply determine whether or not the input is iterable.

Works by trying to iterate via iter(), and if that raises an exception, it’s not iterable.

From http://stackoverflow.com/questions/1952464/in-python-how-do-i-determine-if-an-object-is-iterable

checktype(obj=None, objtype=None, subtype=None, die=False)[source]

A convenience function for checking instances. If objtype is a type, then this function works exactly like isinstance(). But, it can also be one of the following strings:

  • ‘str’, ‘string’: string or bytes object

  • ‘num’, ‘number’: any kind of number

  • ‘arr’, ‘array’: a Numpy array (equivalent to np.ndarray)

  • ‘listlike’: a list, tuple, or array

  • ‘arraylike’: a list, tuple, or array with numeric entries

If subtype is not None, then checktype will iterate over the object and check recursively that each element matches the subtype.

Parameters
  • obj (any) – the object to check the type of

  • objtype (str or type) – the type to confirm the object belongs to

  • subtype (str or type) – optionally check the subtype if the object is iterable

  • die (bool) – whether or not to raise an exception if the object is the wrong type

Examples:

sc.checktype(rand(10), 'array', 'number') # Returns True
sc.checktype(['a','b','c'], 'listlike') # Returns True
sc.checktype(['a','b','c'], 'arraylike') # Returns False
sc.checktype([{'a':3}], list, dict) # Returns True

New in version 2.0.1: pd.Series considered ‘array-like’

isnumber(obj, isnan=None)[source]

Determine whether or not the input is a number.

Parameters
  • obj (any) – the object to check if it’s a number

  • isnan (bool) – an optional additional check to determine whether the number is/isn’t NaN

Almost identical to isinstance(obj, numbers.Number).

isstring(obj)[source]

Determine whether or not the input is string-like (i.e., str or bytes).

Equivalent to isinstance(obj, (str, bytes))

isarray(obj, dtype=None)[source]

Check whether something is a Numpy array, and optionally check the dtype.

Almost the same as isinstance(obj, np.ndarray).

Example:

sc.isarray(np.array([1,2,3]), dtype=float) # False, dtype is int

New in version 1.0.0.

promotetoarray(x, keepnone=False, **kwargs)[source]

Small function to ensure consistent format for things that should be arrays (note: toarray() and promotetoarray() are identical).

Very similar to np.array, with the main difference being that sc.promotetoarray(3) will return np.array([3]) (i.e. a 1-d array that can be iterated over), while np.array(3) will return a 0-d array that can’t be iterated over.

Parameters
  • x (any) – a number or list of numbers

  • keepnone (bool) – whether sc.promotetoarray(None) should return np.array([]) or np.array([None], dtype=object)

  • kwargs (dict) – passed to np.array()

Examples:

sc.promotetoarray(5) # Returns np.array([5])
sc.promotetoarray([3,5]) # Returns np.array([3,5])
sc.promotetoarray(None, skipnone=True) # Returns np.array([])
New in version 1.1.0: replaced “skipnone” with “keepnone”; allowed passing

kwargs to np.array(). | New in version 2.0.1: added support for pandas Series and DataFrame

promotetolist(obj=None, objtype=None, keepnone=False, coerce='default')[source]

Make sure object is always a list (note: tolist()/promotetolist() are identical).

Used so functions can handle inputs like 'a' or ['a', 'b']. In other words, if an argument can either be a single thing (e.g., a single dict key) or a list (e.g., a list of dict keys), this function can be used to do the conversion, so it’s always safe to iterate over the output.

While this usually wraps objects in a list rather than converts them to a list, the “coerce” argument can be used to change this behavior. Options are:

  • ‘none’ or None: do not coerce

  • ‘default’: coerce objects that were lists in Python 2 (range, map, dict_keys, dict_values, dict_items)

  • ‘tuple’: all the types in default, plus tuples

  • ‘full’: all the types in default, plus tuples and arrays

Parameters
  • obj (anything) – object to ensure is a list

  • objtype (anything) – optional type to check for each element; see sc.checktype() for details

  • keepnone (bool) – if keepnone is false, then None is converted to []; else, it’s converted to [None]

  • coerce (str/tuple) – tuple of additional types to coerce to a list (as opposed to wrapping in a list)

Examples:

sc.promotetolist(5) # Returns [5]
sc.promotetolist(np.array([3,5])) # Returns [np.array([3,5])] -- not [3,5]!
sc.promotetolist(np.array([3,5]), coerce=np.ndarray) # Returns [3,5], since arrays are coerced to lists
sc.promotetolist(None) # Returns []
sc.promotetolist(range(3)) # Returns [0,1,2] since range is coerced by default
sc.promotetolist(['a', 'b', 'c'], objtype='number') # Raises exception

def myfunc(data, keys):
    keys = sc.promotetolist(keys)
    for key in keys:
        print(data[key])

data = {'a':[1,2,3], 'b':[4,5,6]}
myfunc(data, keys=['a', 'b']) # Works
myfunc(data, keys='a') # Still works, equivalent to needing to supply keys=['a'] without promotetolist()
New in version 1.1.0: “coerce” argument
New in version 1.2.2: default coerce values
New in version 2.0.2: tuple coersion
toarray(x, keepnone=False, **kwargs)

Small function to ensure consistent format for things that should be arrays (note: toarray() and promotetoarray() are identical).

Very similar to np.array, with the main difference being that sc.promotetoarray(3) will return np.array([3]) (i.e. a 1-d array that can be iterated over), while np.array(3) will return a 0-d array that can’t be iterated over.

Parameters
  • x (any) – a number or list of numbers

  • keepnone (bool) – whether sc.promotetoarray(None) should return np.array([]) or np.array([None], dtype=object)

  • kwargs (dict) – passed to np.array()

Examples:

sc.promotetoarray(5) # Returns np.array([5])
sc.promotetoarray([3,5]) # Returns np.array([3,5])
sc.promotetoarray(None, skipnone=True) # Returns np.array([])
New in version 1.1.0: replaced “skipnone” with “keepnone”; allowed passing

kwargs to np.array(). | New in version 2.0.1: added support for pandas Series and DataFrame

tolist(obj=None, objtype=None, keepnone=False, coerce='default')

Make sure object is always a list (note: tolist()/promotetolist() are identical).

Used so functions can handle inputs like 'a' or ['a', 'b']. In other words, if an argument can either be a single thing (e.g., a single dict key) or a list (e.g., a list of dict keys), this function can be used to do the conversion, so it’s always safe to iterate over the output.

While this usually wraps objects in a list rather than converts them to a list, the “coerce” argument can be used to change this behavior. Options are:

  • ‘none’ or None: do not coerce

  • ‘default’: coerce objects that were lists in Python 2 (range, map, dict_keys, dict_values, dict_items)

  • ‘tuple’: all the types in default, plus tuples

  • ‘full’: all the types in default, plus tuples and arrays

Parameters
  • obj (anything) – object to ensure is a list

  • objtype (anything) – optional type to check for each element; see sc.checktype() for details

  • keepnone (bool) – if keepnone is false, then None is converted to []; else, it’s converted to [None]

  • coerce (str/tuple) – tuple of additional types to coerce to a list (as opposed to wrapping in a list)

Examples:

sc.promotetolist(5) # Returns [5]
sc.promotetolist(np.array([3,5])) # Returns [np.array([3,5])] -- not [3,5]!
sc.promotetolist(np.array([3,5]), coerce=np.ndarray) # Returns [3,5], since arrays are coerced to lists
sc.promotetolist(None) # Returns []
sc.promotetolist(range(3)) # Returns [0,1,2] since range is coerced by default
sc.promotetolist(['a', 'b', 'c'], objtype='number') # Raises exception

def myfunc(data, keys):
    keys = sc.promotetolist(keys)
    for key in keys:
        print(data[key])

data = {'a':[1,2,3], 'b':[4,5,6]}
myfunc(data, keys=['a', 'b']) # Works
myfunc(data, keys='a') # Still works, equivalent to needing to supply keys=['a'] without promotetolist()
New in version 1.1.0: “coerce” argument
New in version 1.2.2: default coerce values
New in version 2.0.2: tuple coersion
transposelist(obj)[source]

Convert e.g. a list of key-value tuples into a list of keys and a list of values.

Example:

o = sc.odict(a=1, b=4, c=9, d=16)
itemlist = o.enumitems()
inds, keys, vals = sc.transposelist(itemlist)

New in version 1.1.0.

swapdict(d)[source]

Swap the keys and values of a dictionary. Equivalent to {v:k for k,v in d.items()}

Parameters

d (dict) – dictionary

Example::

d1 = {‘a’:’foo’, ‘b’:’bar’} d2 = sc.swapdict(d1) # Returns {‘foo’:’a’, ‘bar’:’b’}

New in version 1.3.0.

mergedicts(*args, _strict=False, _overwrite=True, _copy=False, _sameclass=True, _die=True, **kwargs)[source]

Small function to merge multiple dicts together.

By default, skips any input arguments that are None, and allows keys to be set multiple times. This function is similar to dict.update(), except it returns a value. The first dictionary supplied will be used for the output type (e.g. if the first dictionary is an odict, an odict will be returned).

Note that arguments start with underscores to avoid possible collisions with keywords (e.g. sc.mergedicts(dict(loose=True, strict=True), strict=False, _strict=True)).

This function is useful for cases, e.g. function arguments, where the default option is None but you will need a dict later on.

Parameters
  • _strict (bool) – if True, raise an exception if an argument isn’t a dict

  • _overwrite (bool) – if False, raise an exception if multiple keys are found

  • _copy (bool) – whether or not to deepcopy the merged dictionary

  • _sameclass (bool) – whether to ensure the output has the same type as the first dictionary merged

  • _die (bool) – whether to raise an exception if something goes wrong

  • *args (list) – the sequence of dicts to be merged

  • **kwargs (dict) – merge these into the dict as well

Examples:

d0 = sc.mergedicts(user_args) # Useful if user_args might be None, but d0 is always a dict
d1 = sc.mergedicts({'a':1}, {'b':2}) # Returns {'a':1, 'b':2}
d2 = sc.mergedicts({'a':1, 'b':2}, {'b':3, 'c':4}, None) # Returns {'a':1, 'b':3, 'c':4}
d3 = sc.mergedicts(sc.odict({'b':3, 'c':4}), {'a':1, 'b':2}) # Returns sc.odict({'b':2, 'c':4, 'a':1})
d4 = sc.mergedicts({'b':3, 'c':4}, {'a':1, 'b':2}, _overwrite=False) # Raises exception
New in version 1.1.0: “copy” argument
New in version 1.3.3: keywords allowed
New in version 2.0.0: keywords fully enabled; “_sameclass” argument
New in version 2.0.1: fixed bug with “_copy” argument
mergelists(*args, copy=False, **kwargs)[source]

Merge multiple lists together.

Parameters
  • args (any) – the lists, or items, to be joined together into a list

  • copy (bool) – whether to deepcopy the resultant object

  • kwargs (dict) – passed to sc.promotetolist(), which is called on each argument

Examples:

sc.mergelists(None) # Returns []
sc.mergelists([1,2,3], [4,5,6]) # Returns [1, 2, 3, 4, 5, 6]
sc.mergelists([1,2,3], 4, 5, 6) # Returns [1, 2, 3, 4, 5, 6]
sc.mergelists([(1,2), (3,4)], (5,6)) # Returns [(1, 2), (3, 4), (5, 6)]
sc.mergelists((1,2), (3,4), (5,6)) # Returns [(1, 2), (3, 4), (5, 6)]
sc.mergelists((1,2), (3,4), (5,6), coerce=tuple) # Returns [1, 2, 3, 4, 5, 6]

New in version 1.1.0.

strjoin(*args, sep=', ')[source]

Like string join(), but handles more flexible inputs, converts items to strings. By default, join with commas.

Parameters
  • args (list) – the list of items to join

  • sep (str) – the separator string

Example:

sc.strjoin([1,2,3], 4, 'five')

New in version 1.1.0.

newlinejoin(*args)[source]

Alias to strjoin(*args, sep='\n').

Example:

sc.newlinejoin([1,2,3], 4, 'five')

New in version 1.1.0.

strsplit(string, sep=None, skipempty=True, lstrip=True, rstrip=True)[source]

Convenience function to split common types of strings.

Note: to use regular expressions, use re.split() instead.

Parameters
  • string (str) – the string to split

  • sep (str/list) – the types of separator to accept (default space or comma, i.e. [’ ‘, ‘,’])

  • skipempty (bool) – whether to skip empty entries (i.e. from consecutive delimiters)

  • lstrip (bool) – whether to strip any extra spaces on the left

  • rstrip (bool) – whether to strip any extra spaces on the right

Examples

sc.strsplit(‘a b c’) # Returns [‘a’, ‘b’, ‘c’] sc.strsplit(‘a,b,c’) # Returns [‘a’, ‘b’, ‘c’] sc.strsplit(‘a, b, c’) # Returns [‘a’, ‘b’, ‘c’] sc.strsplit(’ foo_bar ‘, sep=’_’) # Returns [‘foo’, ‘bar’]

New in version 2.0.0.

runcommand(command, printinput=False, printoutput=False, wait=True)[source]

Make it easier to run shell commands.

Alias to subprocess.Popen().

Examples:

myfiles = sc.runcommand('ls').split('\n') # Get a list of files in the current folder
sc.runcommand('sshpass -f %s scp myfile.txt me@myserver:myfile.txt' % 'pa55w0rd', printinput=True, printoutput=True) # Copy a file remotely
sc.runcommand('sleep 600; mkdir foo', wait=False) # Waits 10 min, then creates the folder "foo", but the function returns immediately

Date: 2019sep04

gitinfo(path=None, hashlen=7, die=False, verbose=True)[source]

Retrieve git info

This function reads git branch and commit information from a .git directory. Given a path, it will check for a .git directory. If the path doesn’t contain that directory, it will search parent directories for .git until it finds one. Then, the current information will be parsed.

Note: if direct directory reading fails, it will attempt to use the gitpython library.

Parameters
  • path (str) – A folder either containing a .git directory, or with a parent that contains a .git directory

  • hashlen (int) – Length of hash to return (default: 7)

  • die (bool) – whether to raise an exception if git information can’t be retrieved (default: no)

  • verbose (bool) – if not dying, whether to print information about the exception

Returns

Dictionary containing the branch, hash, and commit date

Examples:

info = sc.gitinfo() # Get git info for current script repository
info = sc.gitinfo(my_package.__file__) # Get git info for a particular Python package
compareversions(version1, version2)[source]

Function to compare versions, expecting both arguments to be a string of the format 1.2.3, but numeric works too. Returns 0 for equality, -1 for v1<v2, and 1 for v1>v2.

If version2 starts with >, >=, <, <=, or ==, the function returns True or False depending on the result of the comparison.

Examples:

sc.compareversions('1.2.3', '2.3.4') # returns -1
sc.compareversions(2, '2') # returns 0
sc.compareversions('3.1', '2.99') # returns 1
sc.compareversions('3.1', '>=2.99') # returns True
sc.compareversions(mymodule.__version__, '>=1.0') # common usage pattern
sc.compareversions(mymodule, '>=1.0') # alias to the above

New in version 1.2.1: relational operators

uniquename(name=None, namelist=None, style=None)[source]

Given a name and a list of other names, find a replacement to the name that doesn’t conflict with the other names, and pass it back.

Example:

name = sc.uniquename(name='file', namelist=['file', 'file (1)', 'file (2)'])
suggest(user_input, valid_inputs, n=1, threshold=None, fulloutput=False, die=False, which='damerau')[source]

Return suggested item

Returns item with lowest Levenshtein distance, where case substitution and stripping whitespace are not included in the distance. If there are ties, then the additional operations will be included.

Parameters
  • user_input (str) – User’s input

  • valid_inputs (list) – List/collection of valid strings

  • n (int) – Maximum number of suggestions to return

  • threshold (int) – Maximum number of edits required for an option to be suggested (by default, two-thirds the length of the input; for no threshold, set to -1)

  • fulloutput (bool) – Whether to return suggestions and distances.

  • die (bool) – If True, an informative error will be raised (to avoid having to implement this in the calling code)

  • which (str) – Distance calculation method used; options are “damerau” (default), “levenshtein”, or “jaro”

Returns

Suggested string. Returns None if no suggestions with edit distance less than threshold were found. This helps to make

suggestions more relevant.

Return type

suggestions (str or list)

Examples:

>>> sc.suggest('foo',['Foo','Bar'])
'Foo'
>>> sc.suggest('foo',['FOO','Foo'])
'Foo'
>>> sc.suggest('foo',['Foo ','boo'])
'Foo '
getcaller(frame=2, tostring=True, includelineno=False, includeline=False)[source]

Try to get information on the calling function, but fail gracefully.

Frame 1 is the file calling this function, so not very useful. Frame 2 is the default assuming, it is being called directly. Frame 3 is used if another function is calling this function internally.

Parameters
  • frame (int) – how many frames to descend (e.g. the caller of the caller of the…), default 2

  • tostring (bool) – whether to return a string instead of a dict with filename and line number

  • includelineno (bool) – if tostring, whether to also include the line number

  • includeline (bool) – if not tostring, also store the line contents

Returns

the filename (and line number) of the calling function, either as a string or dict

Return type

output (str/dict)

Examples:

sc.getcaller()
sc.getcaller(tostring=False)['filename'] # Equivalent to sc.getcaller()
sc.getcaller(frame=3) # Descend one level deeper than usual
sc.getcaller(frame=1, tostring=False, includeline=True) # See the line that called sc.getcaller()
New in version 1.0.0.
New in version 1.3.3: do not include line by default
importbyname(module=None, variable=None, namespace=None, lazy=False, overwrite=True, die=True, **kwargs)[source]

Import modules by name.

See https://peps.python.org/pep-0690/ for a proposal for incorporating something similar into Python by default.

Parameters
  • module (str) – name of the module to import

  • variable (str) – the name of the variable to assign the module to (by default, the module’s name)

  • namespace (dict) – the namespace to load the modules into (by default, globals)

  • lazy (bool) – whether to create a LazyModule object instead of load the actual module

  • overwrite (bool) – whether to allow overwriting an existing variable (by default, yes)

  • die (bool) – whether to raise an exception if encountered

  • **kwargs (dict) – additional variable:modules pairs to import (see examples below)

Examples:

np = sc.importbyname('numpy')
sc.importbyname(pd='pandas', np='numpy')
pl = sc.importbyname(pl='matplotlib.pyplot', lazy=True) # Won't actually import until e.g. pl.figure() is called
exception KeyNotFoundError[source]

A tiny class to fix repr for KeyErrors. KeyError prints the repr of the error message, rather than the actual message, so e.g. newline characters print as the character rather than the actual newline.

Example:

raise sc.KeyNotFoundError('The key "foo" is not available, but these are: "bar", "cat"')
exception LinkException[source]

An exception to raise when links are broken, for exclusive use with the Link class.

class prettyobj(*args, **kwargs)[source]

Use pretty repr for objects, instead of just showing the type and memory pointer (the Python default for objects). Can also be used as the base class for custom classes.

Examples

>>> myobj = sc.prettyobj()
>>> myobj.a = 3
>>> myobj.b = {'a':6}
>>> print(myobj)
<sciris.sc_utils.prettyobj at 0x7ffa1e243910>
————————————————————————————————————————————————————————————
a: 3
b: {'a': 6}
————————————————————————————————————————————————————————————
>>> class MyObj(sc.prettyobj):
>>>
>>>     def __init__(self, a, b):
>>>         self.a = a
>>>         self.b = b
>>>
>>>     def mult(self):
>>>         return self.a * self.b
>>>
>>> myobj = MyObj(a=4, b=6)
>>> print(myobj)
<__main__.MyObj at 0x7fd9acd96c10>
————————————————————————————————————————————————————————————
Methods:
  mult()
————————————————————————————————————————————————————————————
a: 4
b: 6
————————————————————————————————————————————————————————————
New in version 2.0.0: allow positional arguments
class autolist(*args)[source]

A simple extension to a list that defines add methods to simplify appending and extension.

Examples:

ls = sc.autolist(3) # Quickly convert a scalar to a list

ls = sc.autolist()
for i in range(5):
    ls += i # No need for ls += [i]

A class to differentiate between an object and a link to an object. The idea is that this object is parsed differently from other objects – most notably, a recursive method (such as a pickle) would skip over Link objects, and then would fix them up after the other objects had been reinstated.

Version: 2017jan31

class LazyModule(module, variable, namespace=None, overwrite=True)[source]

Create a “lazy” module that is loaded if and only if an attribute is called.

Typically not for use by the user, but is used by sc.importbyname().

Parameters
  • module (str) – name of the module to (not) load

  • variable (str) – variable name to assign the module to

  • namespace (dict) – the namespace to use (if not supplied, globals())

  • overwrite (bool) – whether to allow overwriting an existing variable (by default, yes)

Example:

pd = sc.LazyModule('pandas', 'pd') # pd is a LazyModule, not actually pandas
df = pd.DataFrame() # Not only does this work, but pd is now actually pandas

New in version 2.0.0.