sciris.sc_utils¶

Miscellaneous utilities for type checking, printing, dates and times, etc.

Note: there are a lot! The design philosophy has been that it’s easier to ignore a function that you don’t need than write one from scratch that you do need.

Highlights:

dcp(): shortcut to copy.deepcopy()
pp(): shortcut to pprint.pprint()
isnumber(): checks if something is any number type
tolist(): converts any object to a list, for easy iteration
toarray(): tries to convert any object to an array, for easy use with numpy
mergedicts(): merges any set of inputs into a dictionary
mergelists(): merges any set of inputs into a list
runcommand(): simple way of executing a shell command
download(): download multiple URLs in parallel

Functions

`asciify`	Convert an arbitrary Unicode string to ASCII.
`checktype`	A convenience function for checking instances.
`compareversions`	Function to compare versions, expecting both arguments to be a string of the format 1.2.3, but numeric works too.
`cp`	Shortcut to perform a shallow copy operation
`dcp`	Shortcut to perform a deep copy operation
`download`	Download one or more URLs in parallel and return output or save them to disk.
`fast_uuid`	Create a fast UID or set of UIDs.
`flexstr`	Try converting any object to a "regular" string (i.e.
`freeze`	Alias for pip freeze.
`getcaller`	Try to get information on the calling function, but fail gracefully.
`getplatform`	Return the name of the current platform: 'linux', 'windows', 'mac', or 'other'.
`gitinfo`	Retrieve git info
`htmlify`	Convert a string to its HTML representation by converting unicode characters, characters that need to be escaped, and newlines.
`importbyname`	Import modules by name.
`isarray`	Check whether something is a Numpy array, and optionally check the dtype.
`isiterable`	Simply determine whether or not the input is iterable.
`islinux`	Alias to `sc.getplatform('linux')`
`ismac`	Alias to `sc.getplatform('mac')`
`isnumber`	Determine whether or not the input is a number.
`isstring`	Determine whether or not the input is string-like (i.e., str or bytes).
`iswindows`	Alias to `sc.getplatform('windows')`
`mergedicts`	Small function to merge multiple dicts together.
`mergelists`	Merge multiple lists together.
`newlinejoin`	Alias to `strjoin(*args, sep='\n')`.
`pp`	Shortcut for pretty-printing the object.
`promotetoarray`	Small function to ensure consistent format for things that should be arrays (note: `toarray()` and `promotetoarray()` are identical).
`promotetolist`	Make sure object is always a list (note: tolist()/promotetolist() are identical).
`require`	Check whether environment requirements are met.
`runcommand`	Make it easier to run shell commands.
`sha`	Shortcut for the standard hashing (SHA) method
`strjoin`	Like string `join()`, but handles more flexible inputs, converts items to strings.
`strsplit`	Convenience function to split common types of strings.
`suggest`	Return suggested item
`swapdict`	Swap the keys and values of a dictionary.
`toarray`	Small function to ensure consistent format for things that should be arrays (note: `toarray()` and `promotetoarray()` are identical).
`tolist`	Make sure object is always a list (note: tolist()/promotetolist() are identical).
`traceback`	Shortcut for accessing the traceback
`transposelist`	Convert e.g.
`uniquename`	Given a name and a list of other names, find a replacement to the name that doesn't conflict with the other names, and pass it back.
`urlopen`	Download a single URL.
`uuid`	Shortcut for creating a UUID; default is to create a UUID4.
`wget`	Download a single URL.

Classes

`LazyModule`	Create a "lazy" module that is loaded if and only if an attribute is called.
`Link`	A class to differentiate between an object and a link to an object.
`autolist`	A simple extension to a list that defines add methods to simplify appending and extension.
`prettyobj`	Use pretty repr for objects, instead of just showing the type and memory pointer (the Python default for objects).

Exceptions

`KeyNotFoundError`	A tiny class to fix repr for KeyErrors.
`LinkException`	An exception to raise when links are broken, for exclusive use with the Link class.

fast_uuid(which=None, length=None, n=1, secure=False, forcelist=False, safety=1000, recursion=0, recursion_limit=10, verbose=True)[source]¶

Create a fast UID or set of UIDs. Note: for certain applications, sc.uuid() is faster than sc.fast_uuid()!

Parameters

which (str) – the set of characters to choose from (default ascii)
length (int) – length of UID (default 6)
n (int) – number of UIDs to generate
secure (bool) – whether to generate random numbers from sources provided by the operating system
forcelist (bool) – whether or not to return a list even for a single UID (used for recursive calls)
safety (float) – ensure that the space of possible UIDs is at least this much larger than the number requested
recursion (int) – the recursion level of the call (since the function calls itself if not all UIDs are unique)
recursion_limit (int) – Maximum number of times to try regeneraring keys
verbose (bool) – whether to show progress

Returns

a string UID, or a list of string UIDs

Return type

uid (str or list)

Example:

uuids = sc.fast_uuid(n=100) # Generate 100 UUIDs

Inspired by https://stackoverflow.com/questions/2257441/random-string-generation-with-upper-case-letters-and-digits/30038250#30038250

uuid(uid=None, which=None, die=False, tostring=False, length=None, n=1, **kwargs)[source]¶

Shortcut for creating a UUID; default is to create a UUID4. Can also convert a UUID.

Parameters

uid (str or uuid) – if a string, convert to an actual UUID; otherwise, return unchanged
which (int or str) – if int, choose a Python UUID function; otherwise, generate a random alphanumeric string (default 4)
die (bool) – whether to fail for converting a supplied uuid (default False)
tostring (bool) – whether or not to return a string instead of a UUID object (default False)
length (int) – number of characters to trim to, if returning a string
n (int) – number of UUIDs to generate; if n>1, return a list

Returns

the UID object

Return type

uid (UUID or str)

Examples:

sc.uuid() # Alias to uuid.uuid4()
sc.uuid(which='hex') # Creates a length-6 hex string
sc.uuid(which='ascii', length=10, n=50) # Creates 50 UUIDs of length 10 each using the full ASCII character set

dcp(obj, die=True, verbose=True)[source]¶

Shortcut to perform a deep copy operation

Almost identical to copy.deepcopy(), but optionally fall back to copy() if deepcopy fails.

Parameters

die (bool) – if False, fall back to copy()
verbose (bool) – if die is False, then print a warning if deepcopy() fails

New in version 2.0.0: default die=True instead of False

cp(obj, die=True, verbose=True)[source]¶

Shortcut to perform a shallow copy operation

Almost identical to copy.copy(), but optionally allow failures

New in version 2.0.0: default die=True instead of False

pp(obj, jsonify=True, doprint=None, output=False, verbose=False, **kwargs)[source]¶

Shortcut for pretty-printing the object.

Almost identical to pprint.pprint(), but can also be used as an alias for pprint.pformat().

Parameters

obj (any) – object to print
jsonify (bool) – whether to first convert the object to JSON, to handle things like ordered dicts nicely
doprint (bool) – whether to show output (default true)
output (bool) – whether to return output as a string (default false)
verbose (bool) – whether to show warnings when jsonifying the object
kwargs (dict) – passed to pprint.pprint()

New in version 1.3.1: output argument

sha(obj, encoding='utf-8', digest=False)[source]¶

Shortcut for the standard hashing (SHA) method

Equivalent to hashlib.sha224().

Parameters

obj (any) – the object to be hashed; if not a string, converted to one
encoding (str) – the encoding to use
digest (bool) – whether to return the hex digest instead of the hash objet

Example:

sha1 = sc.sha(dict(foo=1, bar=2), digest=True)
sha2 = sc.sha(dict(foo=1, bar=2), digest=True)
sha3 = sc.sha(dict(foo=1, bar=3), digest=True)
assert sha1 == sha2
assert sha2 != sha3

freeze(lower=False)[source]¶

Alias for pip freeze.

Parameters: lower (bool) – convert all keys to lowercase

Example:

assert 'numpy' in sc.freeze() # One way to check for versions

New in version 1.2.2.

require(reqs=None, *args, exact=False, detailed=False, die=True, verbose=True, **kwargs)[source]¶

Check whether environment requirements are met. Alias to pkg_resources.require().

Parameters

reqs (list/dict) – a list of strings, or a dict of package names and versions
args (list) – additional requirements
kwargs (dict) – additional requirements
exact (bool) – use ‘==’ instead of ‘>=’ as the default comparison operator if not specified
detailed (bool) – return a dict of which requirements are/aren’t met
die (bool) – whether to raise an exception if requirements aren’t met
verbose (bool) – print out the exception if it’s not being raised

Examples:

sc.require('numpy')
sc.require(numpy='')
sc.require(reqs={'numpy':'1.19.1', 'matplotlib':'3.2.2'})
sc.require('numpy>=1.19.1', 'matplotlib==3.2.2', die=False)
sc.require(numpy='1.19.1', matplotlib='==4.2.2', die=False, detailed=True)

New in version 1.2.2.

traceback(*args, **kwargs)[source]¶

Shortcut for accessing the traceback

Alias for traceback.format_exc().

getplatform(expected=None, die=False)[source]¶

Return the name of the current platform: ‘linux’, ‘windows’, ‘mac’, or ‘other’. Alias (kind of) to sys.platform.

Parameters

expected (str) – if not None, check if the current platform is this
die (bool) – if True and expected is defined, raise an exception

Example:

sc.getplatform() # Get current name of platform
sc.getplatform('windows', die=True) # Raise an exception if not on Windows

iswindows(die=False)[source]¶: Alias to sc.getplatform('windows')

islinux(die=False)[source]¶: Alias to sc.getplatform('linux')

ismac(die=False)[source]¶: Alias to sc.getplatform('mac')

asciify(string, form='NFKD', encoding='ascii', errors='ignore', **kwargs)[source]¶

Convert an arbitrary Unicode string to ASCII.

Parameters

form (str) – the type of Unicode normalization to use
encoding (str) – the output string to encode to
errors (str) – how to handle errors
kwargs (dict) – passed to string.decode()

Example:: sc.asciify(‘föö→λ ∈ ℝ’) # Returns ‘foo R’

New in version 2.0.1.

urlopen(url, filename=None, save=False, headers=None, params=None, data=None, prefix='http', convert=True, die=False, return_response=False, verbose=False)[source]¶

Download a single URL.

Alias to urllib.request.urlopen(url).read(). See also sc.download() for downloading multiple URLs. Note: sc.urlopen()/sc.wget() are aliases.

Parameters

url (str) – the URL to open, either as GET or POST
filename (str) – if supplied, save to file instead of returning output
save (bool) – if supplied instead of filename, then use the default filename
headers (dict) – a dictionary of headers to pass
params (dict) – a dictionary of parameters to pass to the GET request
data (dict) –
prefix (str) – the string to ensure the URL starts with (else, add it)
convert (bool) – whether to convert from bytes to string
die (bool) – whether to raise an exception if converting to text failed
return_response (bool) – whether to return the response object instead of the output
verbose (bool) – whether to print progress

Examples:

html = sc.urlopen('sciris.org') # Retrieve into variable html
sc.urlopen('http://sciris.org', filename='sciris.html') # Save to file sciris.html
sc.urlopen('http://sciris.org', save=True, headers={'User-Agent':'Custom agent'}) # Save to the default filename (here, sciris.org), with headers

New in version 2.0.0: renamed from wget to urlopen; new arguments
New in version 2.0.1: creates folders by default if they do not exist
New in version 2.0.4: “prefix” argument, e.g. prepend “http://” if not present

wget(url, filename=None, save=False, headers=None, params=None, data=None, prefix='http', convert=True, die=False, return_response=False, verbose=False)¶

Download a single URL.

Alias to urllib.request.urlopen(url).read(). See also sc.download() for downloading multiple URLs. Note: sc.urlopen()/sc.wget() are aliases.

Parameters

url (str) – the URL to open, either as GET or POST
filename (str) – if supplied, save to file instead of returning output
save (bool) – if supplied instead of filename, then use the default filename
headers (dict) – a dictionary of headers to pass
params (dict) – a dictionary of parameters to pass to the GET request
data (dict) –
prefix (str) – the string to ensure the URL starts with (else, add it)
convert (bool) – whether to convert from bytes to string
die (bool) – whether to raise an exception if converting to text failed
return_response (bool) – whether to return the response object instead of the output
verbose (bool) – whether to print progress

Examples:

html = sc.urlopen('sciris.org') # Retrieve into variable html
sc.urlopen('http://sciris.org', filename='sciris.html') # Save to file sciris.html
sc.urlopen('http://sciris.org', save=True, headers={'User-Agent':'Custom agent'}) # Save to the default filename (here, sciris.org), with headers

New in version 2.0.0: renamed from wget to urlopen; new arguments
New in version 2.0.1: creates folders by default if they do not exist
New in version 2.0.4: “prefix” argument, e.g. prepend “http://” if not present

download(url, *args, filename=None, save=True, parallel=True, verbose=True, **kwargs)[source]¶

Download one or more URLs in parallel and return output or save them to disk.

A wrapper for sc.urlopen(), except with save=True by default.

Parameters

url (str/list/dict) – either a single URL, a list of URLs, or a dict of URL:filename pairs
*args (list) – additional URLs to download
filename (str/list) – either a string or a list of the same length as url (if not supplied, return output)
save (bool) – if supplied instead of filename, then use the default filename
parallel (bool) – whether to download multiple URLs in parallel
verbose (bool) – whether to print progress (if verbose=2, print extra detail on each downloaded URL)
**kwargs (dict) – passed to sc.urlopen()

Examples:

html = sc.download('http://sciris.org') # Download a single URL
data = sc.download('http://sciris.org', 'http://covasim.org') # Download two in parallel
sc.download({'http://sciris.org':'sciris.html', 'http://covasim.org':'covasim.html'}) # Downlaod two and save to disk
sc.download(['http://sciris.org', 'http://covasim.org'], filename=['sciris.html', 'covasim.html']) # Ditto

New in version 2.0.0.

htmlify(string, reverse=False, tostring=False)[source]¶

Convert a string to its HTML representation by converting unicode characters, characters that need to be escaped, and newlines. If reverse=True, will convert HTML to string. If tostring=True, will convert the bytestring back to Unicode.

Examples:

output = sc.htmlify('foo&\nbar') # Returns b'foo&amp;<br>bar'
output = sc.htmlify('föö&\nbar', tostring=True) # Returns 'f&#246;&#246;&amp;&nbsp;&nbsp;&nbsp;&nbsp;bar'
output = sc.htmlify('foo&amp;<br>bar', reverse=True) # Returns 'foo&\nbar'

flexstr(arg, force=True)[source]¶: Try converting any object to a “regular” string (i.e. str), but proceed if it fails. Note: this function calls repr() rather than str() to ensure a more robust representation of objects.

isiterable(obj)[source]¶

Simply determine whether or not the input is iterable.

Works by trying to iterate via iter(), and if that raises an exception, it’s not iterable.

From http://stackoverflow.com/questions/1952464/in-python-how-do-i-determine-if-an-object-is-iterable

checktype(obj=None, objtype=None, subtype=None, die=False)[source]¶

A convenience function for checking instances. If objtype is a type, then this function works exactly like isinstance(). But, it can also be one of the following strings:

‘str’, ‘string’: string or bytes object

‘num’, ‘number’: any kind of number

‘arr’, ‘array’: a Numpy array (equivalent to np.ndarray)

‘listlike’: a list, tuple, or array

‘arraylike’: a list, tuple, or array with numeric entries

If subtype is not None, then checktype will iterate over the object and check recursively that each element matches the subtype.

Parameters

obj (any) – the object to check the type of
objtype (str or type) – the type to confirm the object belongs to
subtype (str or type) – optionally check the subtype if the object is iterable
die (bool) – whether or not to raise an exception if the object is the wrong type

Examples:

sc.checktype(rand(10), 'array', 'number') # Returns True
sc.checktype(['a','b','c'], 'listlike') # Returns True
sc.checktype(['a','b','c'], 'arraylike') # Returns False
sc.checktype([{'a':3}], list, dict) # Returns True

New in version 2.0.1: pd.Series considered ‘array-like’

isnumber(obj, isnan=None)[source]¶

Determine whether or not the input is a number.

Parameters

obj (any) – the object to check if it’s a number
isnan (bool) – an optional additional check to determine whether the number is/isn’t NaN

Almost identical to isinstance(obj, numbers.Number).

isstring(obj)[source]¶

Determine whether or not the input is string-like (i.e., str or bytes).

Equivalent to isinstance(obj, (str, bytes))

isarray(obj, dtype=None)[source]¶

Check whether something is a Numpy array, and optionally check the dtype.

Almost the same as isinstance(obj, np.ndarray).

Example:

sc.isarray(np.array([1,2,3]), dtype=float) # False, dtype is int

New in version 1.0.0.

promotetoarray(x, keepnone=False, **kwargs)[source]¶

Small function to ensure consistent format for things that should be arrays (note: toarray() and promotetoarray() are identical).

Very similar to np.array, with the main difference being that sc.promotetoarray(3) will return np.array([3]) (i.e. a 1-d array that can be iterated over), while np.array(3) will return a 0-d array that can’t be iterated over.

Parameters

x (any) – a number or list of numbers
keepnone (bool) – whether sc.promotetoarray(None) should return np.array([]) or np.array([None], dtype=object)
kwargs (dict) – passed to np.array()

Examples:

sc.promotetoarray(5) # Returns np.array([5])
sc.promotetoarray([3,5]) # Returns np.array([3,5])
sc.promotetoarray(None, skipnone=True) # Returns np.array([])

New in version 1.1.0: replaced “skipnone” with “keepnone”; allowed passing

kwargs to np.array(). | New in version 2.0.1: added support for pandas Series and DataFrame

promotetolist(obj=None, objtype=None, keepnone=False, coerce='default')[source]¶

Make sure object is always a list (note: tolist()/promotetolist() are identical).

Used so functions can handle inputs like 'a' or ['a', 'b']. In other words, if an argument can either be a single thing (e.g., a single dict key) or a list (e.g., a list of dict keys), this function can be used to do the conversion, so it’s always safe to iterate over the output.

While this usually wraps objects in a list rather than converts them to a list, the “coerce” argument can be used to change this behavior. Options are:

‘none’ or None: do not coerce
‘default’: coerce objects that were lists in Python 2 (range, map, dict_keys, dict_values, dict_items)
‘tuple’: all the types in default, plus tuples
‘full’: all the types in default, plus tuples and arrays

Parameters

obj (anything) – object to ensure is a list
objtype (anything) – optional type to check for each element; see sc.checktype() for details
keepnone (bool) – if keepnone is false, then None is converted to []; else, it’s converted to [None]
coerce (str/tuple) – tuple of additional types to coerce to a list (as opposed to wrapping in a list)

Examples:

sc.promotetolist(5) # Returns [5]
sc.promotetolist(np.array([3,5])) # Returns [np.array([3,5])] -- not [3,5]!
sc.promotetolist(np.array([3,5]), coerce=np.ndarray) # Returns [3,5], since arrays are coerced to lists
sc.promotetolist(None) # Returns []
sc.promotetolist(range(3)) # Returns [0,1,2] since range is coerced by default
sc.promotetolist(['a', 'b', 'c'], objtype='number') # Raises exception

def myfunc(data, keys):
    keys = sc.promotetolist(keys)
    for key in keys:
        print(data[key])

data = {'a':[1,2,3], 'b':[4,5,6]}
myfunc(data, keys=['a', 'b']) # Works
myfunc(data, keys='a') # Still works, equivalent to needing to supply keys=['a'] without promotetolist()

New in version 1.1.0: “coerce” argument
New in version 1.2.2: default coerce values
New in version 2.0.2: tuple coersion

toarray(x, keepnone=False, **kwargs)¶

Small function to ensure consistent format for things that should be arrays (note: toarray() and promotetoarray() are identical).

Very similar to np.array, with the main difference being that sc.promotetoarray(3) will return np.array([3]) (i.e. a 1-d array that can be iterated over), while np.array(3) will return a 0-d array that can’t be iterated over.

Parameters

x (any) – a number or list of numbers
keepnone (bool) – whether sc.promotetoarray(None) should return np.array([]) or np.array([None], dtype=object)
kwargs (dict) – passed to np.array()

Examples:

sc.promotetoarray(5) # Returns np.array([5])
sc.promotetoarray([3,5]) # Returns np.array([3,5])
sc.promotetoarray(None, skipnone=True) # Returns np.array([])

New in version 1.1.0: replaced “skipnone” with “keepnone”; allowed passing

kwargs to np.array(). | New in version 2.0.1: added support for pandas Series and DataFrame

tolist(obj=None, objtype=None, keepnone=False, coerce='default')¶

Make sure object is always a list (note: tolist()/promotetolist() are identical).

Used so functions can handle inputs like 'a' or ['a', 'b']. In other words, if an argument can either be a single thing (e.g., a single dict key) or a list (e.g., a list of dict keys), this function can be used to do the conversion, so it’s always safe to iterate over the output.

While this usually wraps objects in a list rather than converts them to a list, the “coerce” argument can be used to change this behavior. Options are:

‘none’ or None: do not coerce
‘default’: coerce objects that were lists in Python 2 (range, map, dict_keys, dict_values, dict_items)
‘tuple’: all the types in default, plus tuples
‘full’: all the types in default, plus tuples and arrays

Parameters

obj (anything) – object to ensure is a list
objtype (anything) – optional type to check for each element; see sc.checktype() for details
keepnone (bool) – if keepnone is false, then None is converted to []; else, it’s converted to [None]
coerce (str/tuple) – tuple of additional types to coerce to a list (as opposed to wrapping in a list)

Examples:

sc.promotetolist(5) # Returns [5]
sc.promotetolist(np.array([3,5])) # Returns [np.array([3,5])] -- not [3,5]!
sc.promotetolist(np.array([3,5]), coerce=np.ndarray) # Returns [3,5], since arrays are coerced to lists
sc.promotetolist(None) # Returns []
sc.promotetolist(range(3)) # Returns [0,1,2] since range is coerced by default
sc.promotetolist(['a', 'b', 'c'], objtype='number') # Raises exception

def myfunc(data, keys):
    keys = sc.promotetolist(keys)
    for key in keys:
        print(data[key])

data = {'a':[1,2,3], 'b':[4,5,6]}
myfunc(data, keys=['a', 'b']) # Works
myfunc(data, keys='a') # Still works, equivalent to needing to supply keys=['a'] without promotetolist()

New in version 1.1.0: “coerce” argument
New in version 1.2.2: default coerce values
New in version 2.0.2: tuple coersion

transposelist(obj)[source]¶

Convert e.g. a list of key-value tuples into a list of keys and a list of values.

Example:

o = sc.odict(a=1, b=4, c=9, d=16)
itemlist = o.enumitems()
inds, keys, vals = sc.transposelist(itemlist)

New in version 1.1.0.

swapdict(d)[source]¶

Swap the keys and values of a dictionary. Equivalent to {v:k for k,v in d.items()}

Parameters: d (dict) – dictionary

Example::: d1 = {‘a’:’foo’, ‘b’:’bar’} d2 = sc.swapdict(d1) # Returns {‘foo’:’a’, ‘bar’:’b’}

New in version 1.3.0.

mergedicts(*args, _strict=False, _overwrite=True, _copy=False, _sameclass=True, _die=True, **kwargs)[source]¶

Small function to merge multiple dicts together.

By default, skips any input arguments that are None, and allows keys to be set multiple times. This function is similar to dict.update(), except it returns a value. The first dictionary supplied will be used for the output type (e.g. if the first dictionary is an odict, an odict will be returned).

Note that arguments start with underscores to avoid possible collisions with keywords (e.g. sc.mergedicts(dict(loose=True, strict=True), strict=False, _strict=True)).

This function is useful for cases, e.g. function arguments, where the default option is None but you will need a dict later on.

Parameters

_strict (bool) – if True, raise an exception if an argument isn’t a dict
_overwrite (bool) – if False, raise an exception if multiple keys are found
_copy (bool) – whether or not to deepcopy the merged dictionary
_sameclass (bool) – whether to ensure the output has the same type as the first dictionary merged
_die (bool) – whether to raise an exception if something goes wrong
*args (list) – the sequence of dicts to be merged
**kwargs (dict) – merge these into the dict as well

Examples:

d0 = sc.mergedicts(user_args) # Useful if user_args might be None, but d0 is always a dict
d1 = sc.mergedicts({'a':1}, {'b':2}) # Returns {'a':1, 'b':2}
d2 = sc.mergedicts({'a':1, 'b':2}, {'b':3, 'c':4}, None) # Returns {'a':1, 'b':3, 'c':4}
d3 = sc.mergedicts(sc.odict({'b':3, 'c':4}), {'a':1, 'b':2}) # Returns sc.odict({'b':2, 'c':4, 'a':1})
d4 = sc.mergedicts({'b':3, 'c':4}, {'a':1, 'b':2}, _overwrite=False) # Raises exception

New in version 1.1.0: “copy” argument
New in version 1.3.3: keywords allowed
New in version 2.0.0: keywords fully enabled; “_sameclass” argument
New in version 2.0.1: fixed bug with “_copy” argument

mergelists(*args, copy=False, **kwargs)[source]¶

Merge multiple lists together.

Parameters

args (any) – the lists, or items, to be joined together into a list
copy (bool) – whether to deepcopy the resultant object
kwargs (dict) – passed to sc.promotetolist(), which is called on each argument

Examples:

sc.mergelists(None) # Returns []
sc.mergelists([1,2,3], [4,5,6]) # Returns [1, 2, 3, 4, 5, 6]
sc.mergelists([1,2,3], 4, 5, 6) # Returns [1, 2, 3, 4, 5, 6]
sc.mergelists([(1,2), (3,4)], (5,6)) # Returns [(1, 2), (3, 4), (5, 6)]
sc.mergelists((1,2), (3,4), (5,6)) # Returns [(1, 2), (3, 4), (5, 6)]
sc.mergelists((1,2), (3,4), (5,6), coerce=tuple) # Returns [1, 2, 3, 4, 5, 6]

New in version 1.1.0.

strjoin(*args, sep=', ')[source]¶

Like string join(), but handles more flexible inputs, converts items to strings. By default, join with commas.

Parameters

args (list) – the list of items to join
sep (str) – the separator string

Example:

sc.strjoin([1,2,3], 4, 'five')

New in version 1.1.0.

newlinejoin(*args)[source]¶

Alias to strjoin(*args, sep='\n').

Example:

sc.newlinejoin([1,2,3], 4, 'five')

New in version 1.1.0.

strsplit(string, sep=None, skipempty=True, lstrip=True, rstrip=True)[source]¶

Convenience function to split common types of strings.

Note: to use regular expressions, use re.split() instead.

Parameters

string (str) – the string to split
sep (str/list) – the types of separator to accept (default space or comma, i.e. [’ ‘, ‘,’])
skipempty (bool) – whether to skip empty entries (i.e. from consecutive delimiters)
lstrip (bool) – whether to strip any extra spaces on the left
rstrip (bool) – whether to strip any extra spaces on the right

Examples

sc.strsplit(‘a b c’) # Returns [‘a’, ‘b’, ‘c’] sc.strsplit(‘a,b,c’) # Returns [‘a’, ‘b’, ‘c’] sc.strsplit(‘a, b, c’) # Returns [‘a’, ‘b’, ‘c’] sc.strsplit(’ foo_bar ‘, sep=’_’) # Returns [‘foo’, ‘bar’]

New in version 2.0.0.

runcommand(command, printinput=False, printoutput=False, wait=True)[source]¶

Make it easier to run shell commands.

Alias to subprocess.Popen().

Examples:

myfiles = sc.runcommand('ls').split('\n') # Get a list of files in the current folder
sc.runcommand('sshpass -f %s scp myfile.txt me@myserver:myfile.txt' % 'pa55w0rd', printinput=True, printoutput=True) # Copy a file remotely
sc.runcommand('sleep 600; mkdir foo', wait=False) # Waits 10 min, then creates the folder "foo", but the function returns immediately

Date: 2019sep04

gitinfo(path=None, hashlen=7, die=False, verbose=True)[source]¶

Retrieve git info

This function reads git branch and commit information from a .git directory. Given a path, it will check for a .git directory. If the path doesn’t contain that directory, it will search parent directories for .git until it finds one. Then, the current information will be parsed.

Note: if direct directory reading fails, it will attempt to use the gitpython library.

Parameters

path (str) – A folder either containing a .git directory, or with a parent that contains a .git directory
hashlen (int) – Length of hash to return (default: 7)
die (bool) – whether to raise an exception if git information can’t be retrieved (default: no)
verbose (bool) – if not dying, whether to print information about the exception

Returns

Dictionary containing the branch, hash, and commit date

Examples:

info = sc.gitinfo() # Get git info for current script repository
info = sc.gitinfo(my_package.__file__) # Get git info for a particular Python package

compareversions(version1, version2)[source]¶

Function to compare versions, expecting both arguments to be a string of the format 1.2.3, but numeric works too. Returns 0 for equality, -1 for v1<v2, and 1 for v1>v2.

If version2 starts with >, >=, <, <=, or ==, the function returns True or False depending on the result of the comparison.

Examples:

sc.compareversions('1.2.3', '2.3.4') # returns -1
sc.compareversions(2, '2') # returns 0
sc.compareversions('3.1', '2.99') # returns 1
sc.compareversions('3.1', '>=2.99') # returns True
sc.compareversions(mymodule.__version__, '>=1.0') # common usage pattern
sc.compareversions(mymodule, '>=1.0') # alias to the above

New in version 1.2.1: relational operators

uniquename(name=None, namelist=None, style=None)[source]¶

Given a name and a list of other names, find a replacement to the name that doesn’t conflict with the other names, and pass it back.

Example:

name = sc.uniquename(name='file', namelist=['file', 'file (1)', 'file (2)'])

suggest(user_input, valid_inputs, n=1, threshold=None, fulloutput=False, die=False, which='damerau')[source]¶

Return suggested item

Returns item with lowest Levenshtein distance, where case substitution and stripping whitespace are not included in the distance. If there are ties, then the additional operations will be included.

Parameters

user_input (str) – User’s input
valid_inputs (list) – List/collection of valid strings
n (int) – Maximum number of suggestions to return
threshold (int) – Maximum number of edits required for an option to be suggested (by default, two-thirds the length of the input; for no threshold, set to -1)
fulloutput (bool) – Whether to return suggestions and distances.
die (bool) – If True, an informative error will be raised (to avoid having to implement this in the calling code)
which (str) – Distance calculation method used; options are “damerau” (default), “levenshtein”, or “jaro”

Returns

Suggested string. Returns None if no suggestions with edit distance less than threshold were found. This helps to make: suggestions more relevant.

Return type

suggestions (str or list)

Examples:

>>> sc.suggest('foo',['Foo','Bar'])
'Foo'
>>> sc.suggest('foo',['FOO','Foo'])
'Foo'
>>> sc.suggest('foo',['Foo ','boo'])
'Foo '

getcaller(frame=2, tostring=True, includelineno=False, includeline=False)[source]¶

Try to get information on the calling function, but fail gracefully.

Frame 1 is the file calling this function, so not very useful. Frame 2 is the default assuming, it is being called directly. Frame 3 is used if another function is calling this function internally.

Parameters

frame (int) – how many frames to descend (e.g. the caller of the caller of the…), default 2
tostring (bool) – whether to return a string instead of a dict with filename and line number
includelineno (bool) – if tostring, whether to also include the line number
includeline (bool) – if not tostring, also store the line contents

Returns

the filename (and line number) of the calling function, either as a string or dict

Return type

output (str/dict)

Examples:

sc.getcaller()
sc.getcaller(tostring=False)['filename'] # Equivalent to sc.getcaller()
sc.getcaller(frame=3) # Descend one level deeper than usual
sc.getcaller(frame=1, tostring=False, includeline=True) # See the line that called sc.getcaller()

New in version 1.0.0.

New in version 1.3.3: do not include line by default

importbyname(module=None, variable=None, namespace=None, lazy=False, overwrite=True, die=True, **kwargs)[source]¶

Import modules by name.

See https://peps.python.org/pep-0690/ for a proposal for incorporating something similar into Python by default.

Parameters

module (str) – name of the module to import
variable (str) – the name of the variable to assign the module to (by default, the module’s name)
namespace (dict) – the namespace to load the modules into (by default, globals)
lazy (bool) – whether to create a LazyModule object instead of load the actual module
overwrite (bool) – whether to allow overwriting an existing variable (by default, yes)
die (bool) – whether to raise an exception if encountered
**kwargs (dict) – additional variable:modules pairs to import (see examples below)

Examples:

np = sc.importbyname('numpy')
sc.importbyname(pd='pandas', np='numpy')
pl = sc.importbyname(pl='matplotlib.pyplot', lazy=True) # Won't actually import until e.g. pl.figure() is called

exception KeyNotFoundError[source]¶

A tiny class to fix repr for KeyErrors. KeyError prints the repr of the error message, rather than the actual message, so e.g. newline characters print as the character rather than the actual newline.

Example:

raise sc.KeyNotFoundError('The key "foo" is not available, but these are: "bar", "cat"')

exception LinkException[source]¶: An exception to raise when links are broken, for exclusive use with the Link class.

class prettyobj(*args, **kwargs)[source]¶

Use pretty repr for objects, instead of just showing the type and memory pointer (the Python default for objects). Can also be used as the base class for custom classes.

Examples

>>> myobj = sc.prettyobj()
>>> myobj.a = 3
>>> myobj.b = {'a':6}
>>> print(myobj)
<sciris.sc_utils.prettyobj at 0x7ffa1e243910>
————————————————————————————————————————————————————————————
a: 3
b: {'a': 6}
————————————————————————————————————————————————————————————

>>> class MyObj(sc.prettyobj):
>>>
>>>     def __init__(self, a, b):
>>>         self.a = a
>>>         self.b = b
>>>
>>>     def mult(self):
>>>         return self.a * self.b
>>>
>>> myobj = MyObj(a=4, b=6)
>>> print(myobj)
<__main__.MyObj at 0x7fd9acd96c10>
————————————————————————————————————————————————————————————
Methods:
  mult()
————————————————————————————————————————————————————————————
a: 4
b: 6
————————————————————————————————————————————————————————————

New in version 2.0.0: allow positional arguments

class autolist(*args)[source]¶

A simple extension to a list that defines add methods to simplify appending and extension.

Examples:

ls = sc.autolist(3) # Quickly convert a scalar to a list

ls = sc.autolist()
for i in range(5):
    ls += i # No need for ls += [i]

class Link(obj=None)[source]¶

A class to differentiate between an object and a link to an object. The idea is that this object is parsed differently from other objects – most notably, a recursive method (such as a pickle) would skip over Link objects, and then would fix them up after the other objects had been reinstated.

Version: 2017jan31

class LazyModule(module, variable, namespace=None, overwrite=True)[source]¶

Create a “lazy” module that is loaded if and only if an attribute is called.

Typically not for use by the user, but is used by sc.importbyname().

Parameters

module (str) – name of the module to (not) load
variable (str) – variable name to assign the module to
namespace (dict) – the namespace to use (if not supplied, globals())
overwrite (bool) – whether to allow overwriting an existing variable (by default, yes)

Example:

pd = sc.LazyModule('pandas', 'pd') # pd is a LazyModule, not actually pandas
df = pd.DataFrame() # Not only does this work, but pd is now actually pandas

New in version 2.0.0.