Advanced features#

Here are yet more tools that the average user won’t need, but might come in handy one day.

Click here to open an interactive version of this notebook.

Nested dictionaries#

Nested dictionaries are a useful way of storing complex data (and in fact are more or less the basis of JSON), but can be a pain to interact with if you don’t know the structure in advance. Sciris has several functions for working with nested dictionaries. For example:

import sciris as sc

# Create the structure
nest = {}
sc.makenested(nest, ['key1','key1.1'])
sc.makenested(nest, ['key1','key1.2'])
sc.makenested(nest, ['key1','key1.3'])
sc.makenested(nest, ['key2','key2.1','key2.1.1'])
sc.makenested(nest, ['key2','key2.2','key2.2.1'])

# Set the value for each "twig"
count = 0
for twig in sc.iternested(nest):
    count += 1
    sc.setnested(nest, twig, count)

# Convert to a JSON to view the structure more clearly

# Get all the values from the dict
values = []
for twig in sc.iternested(nest):
    values.append(sc.getnested(nest, twig))
print(f'{values = }')
  "key1": {
    "key1.1": 1,
    "key1.2": 2,
    "key1.3": 3
  "key2": {
    "key2.1": {
      "key2.1.1": 4
    "key2.2": {
      "key2.2.1": 5
values = [1, 2, 3, 4, 5]

Sciris also has a function, which can find either keys or values that match a certain pattern:

print(, 'key2.1.1'))
print(, value=5))
[['key2', 'key2.1', 'key2.1.1']]
[['key2', 'key2.2', 'key2.2.1']]

There’s even an sc.iterobj() function that can make arbitrary changes to an object:

def increment(obj):
    return obj + 1000 if isinstance(obj, int) and obj !=3 else obj

sc.iterobj(nest, increment, inplace=True)
  "key1": {
    "key1.1": 1001,
    "key1.2": 1002,
    "key1.3": 3
  "key2": {
    "key2.1": {
      "key2.1.1": 1004
    "key2.2": {
      "key2.2.1": 1005

Context blocks#

Sciris contains two context block (i.e. “with ... as”) classes for catching what happens inside them.

sc.capture() captures all text output to a variable:

import sciris as sc
import numpy as np

def verbose_func(n=200):
    for i in range(n):
        print(f'Here are 5 random numbers: {np.random.rand(5)}')

with sc.capture() as text:

lines = text.splitlines()
target = '777'
for l,line in enumerate(lines):
    if target in line:
        print(f'Found target {target} on line {l}: {line}')
Found target 777 on line 37: Here are 5 random numbers: [0.23091441 0.92720101 0.64386704 0.98108182 0.77739743]
Found target 777 on line 40: Here are 5 random numbers: [0.10017502 0.85447773 0.70585521 0.79761481 0.56303318]
Found target 777 on line 42: Here are 5 random numbers: [0.98338691 0.23776133 0.79377702 0.38521457 0.13102052]
Found target 777 on line 57: Here are 5 random numbers: [0.48547204 0.0794115  0.83758695 0.59309727 0.48170777]
Found target 777 on line 128: Here are 5 random numbers: [0.91227649 0.17771961 0.64710676 0.36414018 0.0829745 ]

The other function, sc.tryexcept(), is a more compact way of writing try ... except blocks, and gives detailed control of error handling:

def fickle_func(n=1):
    for i in range(n):
        rnd = np.random.rand()
        if rnd < 0.005:
            raise ValueError(f'Value {rnd:n} too small')
        elif rnd > 0.99:
            raise RuntimeError(f'Value {rnd:n} too big')

sc.heading('Simple usage, exit gracefully at first exception')
with sc.tryexcept():

sc.heading('Store all history')
tryexc = None
for i in range(1000):
    with sc.tryexcept(history=tryexc, verbose=False) as tryexc:

Simple usage, exit gracefully at first exception

<class 'RuntimeError'> Value 0.995604 too big

Store all history

                      type                        value                             traceback
0   <class 'RuntimeError'>       Value 0.992427 too big  <traceback object at 0x7f3430157a80>
1     <class 'ValueError'>    Value 0.0035279 too small  <traceback object at 0x7f3428df4a80>
2   <class 'RuntimeError'>       Value 0.991026 too big  <traceback object at 0x7f3402fa3a80>
3   <class 'RuntimeError'>       Value 0.996673 too big  <traceback object at 0x7f3402eb74c0>
4     <class 'ValueError'>  Value 0.000397334 too small  <traceback object at 0x7f3402eb7400>
5   <class 'RuntimeError'>       Value 0.992735 too big  <traceback object at 0x7f3402eb7140>
6   <class 'RuntimeError'>       Value 0.995062 too big  <traceback object at 0x7f3402eb76c0>
7   <class 'RuntimeError'>       Value 0.992525 too big  <traceback object at 0x7f3402eb7540>
8   <class 'RuntimeError'>       Value 0.993638 too big  <traceback object at 0x7f3402eb6f00>
9     <class 'ValueError'>     Value 0.001585 too small  <traceback object at 0x7f3402eb7800>
10    <class 'ValueError'>   Value 0.00207436 too small  <traceback object at 0x7f3402eb7300>
11  <class 'RuntimeError'>       Value 0.996381 too big  <traceback object at 0x7f3402eb7880>

Interpolation and optimization#

Sciris includes two algorithms that complement their SciPy relatives: interpolation and optimization.


The function sc.smoothinterp() smoothly interpolates between points but does not use spline interpolation; this makes it somewhat of a balance between numpy.interp() (which only interpolates linearly) and scipy.interpolate.interp1d(..., method='cubic'), which takes considerable liberties between data points:

import sciris as sc
import numpy as np
import pylab as pl
from scipy import interpolate

# Create the data
origy = np.array([0, 0.2, 0.1, 0.9, 0.7, 0.8, 0.95, 1])
origx = np.linspace(0, 1, len(origy))
newx  = np.linspace(0, 1)

# Create the interpolations
sc_y = sc.smoothinterp(newx, origx, origy, smoothness=5)
np_y = np.interp(newx, origx, origy)
si_y = interpolate.interp1d(origx, origy, 'cubic')(newx)

# Plot
kw = dict(lw=2, alpha=0.7)
pl.plot(newx, np_y, '--', label='NumPy', **kw)
pl.plot(newx, si_y, ':',  label='SciPy', **kw)
pl.plot(newx, sc_y, '-',  label='Sciris', **kw)
pl.scatter(origx, origy, s=50, c='k', label='Data')

As you can see, sc.smoothinterp() gives a more “reasonable” approximation to the data, at the expense of not exactly passing through all the data points.


Sciris includes a gradient descent optimization method, adaptive stochastic descent (ASD), that can outperform SciPy’s built-in optimization methods (such as simplex) for certain types of optimization problem. For example:

# Basic usage
import numpy as np
import sciris as sc
from scipy import optimize

# Very simple optimization problem -- set all numbers to 0
func = np.linalg.norm
x = [1, 2, 3]

with sc.timer('scipy.optimize()'):
    opt_scipy = optimize.minimize(func, x)

with sc.timer('sciris.asd()'):
    opt_sciris = sc.asd(func, x, verbose=False)

print(f'Scipy result:  {func(opt_scipy.x)}')
print(f'Sciris result: {func(opt_sciris.x)}')
scipy.optimize(): 21.5 ms
sciris.asd(): 5.48 ms
Scipy result:  4.829290232718364e-08
Sciris result: 2.9796874070320123e-16

Compared to SciPy’s simplex algorithm, Sciris’ ASD algorithm was ≈3 times faster and found a result ≈8 orders of magnitude smaller.


And finally, let’s end on something fun. Sciris has an sc.animation() class with lots of options, but you can also just make a quick movie from a series of plots. For example, let’s make some lines dance:

frames = [pl.plot(pl.cumsum(pl.randn(100))) for i in range(20)] # Create frames
sc.savemovie(frames, 'dancing_lines.gif'); # Save movie as a gif
Saving 20 frames at 10.0 fps and 150 dpi to "dancing_lines.gif" using imagemagick...
Done; movie saved to "dancing_lines.gif"
File size: 127 KB
Elapsed time: 5.50 s

This creates the following movie, which is a rather delightful way to end:


We hope you enjoyed this series of tutorials! Remember, write to us if you want to get in touch.