Advanced features#
Here are yet more tools that the average user won’t need, but might come in handy one day.
Click here to open an interactive version of this notebook.
Nested dictionaries#
Nested dictionaries are a useful way of storing complex data (and in fact are more or less the basis of JSON), but can be a pain to interact with if you don’t know the structure in advance. Sciris has several functions for working with nested dictionaries. For example:
[1]:
import sciris as sc
# Create the structure
nest = {}
sc.makenested(nest, ['key1','key1.1'])
sc.makenested(nest, ['key1','key1.2'])
sc.makenested(nest, ['key1','key1.3'])
sc.makenested(nest, ['key2','key2.1','key2.1.1'])
sc.makenested(nest, ['key2','key2.2','key2.2.1'])
# Set the value for each "twig"
count = 0
for twig in sc.iternested(nest):
count += 1
sc.setnested(nest, twig, count)
# Convert to a JSON to view the structure more clearly
sc.printjson(nest)
# Get all the values from the dict
values = []
for twig in sc.iternested(nest):
values.append(sc.getnested(nest, twig))
print(f'{values = }')
{
"key1": {
"key1.1": 1,
"key1.2": 2,
"key1.3": 3
},
"key2": {
"key2.1": {
"key2.1.1": 4
},
"key2.2": {
"key2.2.1": 5
}
}
}
values = [1, 2, 3, 4, 5]
Sciris also has a sc.search()
function, which can find either keys or values that match a certain pattern:
[2]:
print(sc.search(nest, 'key2.1.1'))
print(sc.search(nest, value=5))
[['key2', 'key2.1', 'key2.1.1']]
[['key2', 'key2.2', 'key2.2.1']]
There’s even an sc.iterobj()
function that can make arbitrary changes to an object:
[3]:
def increment(obj):
return obj + 1000 if isinstance(obj, int) and obj !=3 else obj
sc.iterobj(nest, increment, inplace=True)
sc.printjson(nest)
{
"key1": {
"key1.1": 1001,
"key1.2": 1002,
"key1.3": 3
},
"key2": {
"key2.1": {
"key2.1.1": 1004
},
"key2.2": {
"key2.2.1": 1005
}
}
}
Context blocks#
Sciris contains two context block (i.e. “with ... as
”) classes for catching what happens inside them.
sc.capture()
captures all text output to a variable:
[4]:
import sciris as sc
import numpy as np
def verbose_func(n=200):
for i in range(n):
print(f'Here are 5 random numbers: {np.random.rand(5)}')
with sc.capture() as text:
verbose_func()
lines = text.splitlines()
target = '777'
for l,line in enumerate(lines):
if target in line:
print(f'Found target {target} on line {l}: {line}')
Found target 777 on line 35: Here are 5 random numbers: [0.30647147 0.51483255 0.77745685 0.30184163 0.30430842]
Found target 777 on line 88: Here are 5 random numbers: [0.70554215 0.47936335 0.05371795 0.39730575 0.88777415]
Found target 777 on line 105: Here are 5 random numbers: [0.53431613 0.32916132 0.89777977 0.43933727 0.54951866]
Found target 777 on line 116: Here are 5 random numbers: [0.05608838 0.66205077 0.49744291 0.97772645 0.68897064]
The other function, sc.tryexcept()
, is a more compact way of writing try ... except
blocks, and gives detailed control of error handling:
[5]:
def fickle_func(n=1):
for i in range(n):
rnd = np.random.rand()
if rnd < 0.005:
raise ValueError(f'Value {rnd:n} too small')
elif rnd > 0.99:
raise RuntimeError(f'Value {rnd:n} too big')
sc.heading('Simple usage, exit gracefully at first exception')
with sc.tryexcept():
fickle_func(n=1000)
sc.heading('Store all history')
tryexc = None
for i in range(1000):
with sc.tryexcept(history=tryexc, verbose=False) as tryexc:
fickle_func()
tryexc.disp()
————————————————————————————————————————————————
Simple usage, exit gracefully at first exception
————————————————————————————————————————————————
<class 'RuntimeError'> Value 0.993257 too big
—————————————————
Store all history
—————————————————
type value traceback
0 <class 'RuntimeError'> Value 0.997336 too big <traceback object at 0x7f905c117c80>
1 <class 'RuntimeError'> Value 0.99658 too big <traceback object at 0x7f9030653380>
2 <class 'ValueError'> Value 0.00171662 too small <traceback object at 0x7f9030653000>
3 <class 'RuntimeError'> Value 0.99115 too big <traceback object at 0x7f9030653540>
4 <class 'RuntimeError'> Value 0.999809 too big <traceback object at 0x7f9030653440>
5 <class 'RuntimeError'> Value 0.991638 too big <traceback object at 0x7f9030652840>
6 <class 'RuntimeError'> Value 0.997487 too big <traceback object at 0x7f9030653dc0>
7 <class 'RuntimeError'> Value 0.998563 too big <traceback object at 0x7f9030653740>
8 <class 'RuntimeError'> Value 0.994992 too big <traceback object at 0x7f90306521c0>
9 <class 'RuntimeError'> Value 0.997475 too big <traceback object at 0x7f9030651900>
10 <class 'RuntimeError'> Value 0.999894 too big <traceback object at 0x7f9030653fc0>
Interpolation and optimization#
Sciris includes two algorithms that complement their SciPy relatives: interpolation and optimization.
Interpolation#
The function sc.smoothinterp()
smoothly interpolates between points but does not use spline interpolation; this makes it somewhat of a balance between numpy.interp()
(which only interpolates linearly) and scipy.interpolate.interp1d(..., method='cubic')
, which takes considerable liberties between data points:
[6]:
import sciris as sc
import numpy as np
import pylab as pl
from scipy import interpolate
# Create the data
origy = np.array([0, 0.2, 0.1, 0.9, 0.7, 0.8, 0.95, 1])
origx = np.linspace(0, 1, len(origy))
newx = np.linspace(0, 1)
# Create the interpolations
sc_y = sc.smoothinterp(newx, origx, origy, smoothness=5)
np_y = np.interp(newx, origx, origy)
si_y = interpolate.interp1d(origx, origy, 'cubic')(newx)
# Plot
kw = dict(lw=2, alpha=0.7)
pl.plot(newx, np_y, '--', label='NumPy', **kw)
pl.plot(newx, si_y, ':', label='SciPy', **kw)
pl.plot(newx, sc_y, '-', label='Sciris', **kw)
pl.scatter(origx, origy, s=50, c='k', label='Data')
pl.legend();

As you can see, sc.smoothinterp()
gives a more “reasonable” approximation to the data, at the expense of not exactly passing through all the data points.
Optimization#
Sciris includes a gradient descent optimization method, adaptive stochastic descent (ASD), that can outperform SciPy’s built-in optimization methods (such as simplex) for certain types of optimization problem. For example:
[7]:
# Basic usage
import numpy as np
import sciris as sc
from scipy import optimize
# Very simple optimization problem -- set all numbers to 0
func = np.linalg.norm
x = [1, 2, 3]
with sc.timer('scipy.optimize()'):
opt_scipy = optimize.minimize(func, x)
with sc.timer('sciris.asd()'):
opt_sciris = sc.asd(func, x, verbose=False)
print(f'Scipy result: {func(opt_scipy.x)}')
print(f'Sciris result: {func(opt_sciris.x)}')
scipy.optimize(): 13.3 ms
sciris.asd(): 4.38 ms
Scipy result: 4.829290232718364e-08
Sciris result: 2.897767167584095e-16
Compared to SciPy’s simplex algorithm, Sciris’ ASD algorithm was ≈3 times faster and found a result ≈8 orders of magnitude smaller.
Animation#
And finally, let’s end on something fun. Sciris has an sc.animation()
class with lots of options, but you can also just make a quick movie from a series of plots. For example, let’s make some lines dance:
[8]:
pl.figure()
frames = [pl.plot(pl.cumsum(pl.randn(100))) for i in range(20)] # Create frames
sc.savemovie(frames, 'dancing_lines.gif'); # Save movie as a gif
Saving 20 frames at 10.0 fps and 150 dpi to "dancing_lines.gif" using imagemagick...
Done; movie saved to "dancing_lines.gif"
File size: 150 KB
Elapsed time: 6.06 s

This creates the following movie, which is a rather delightful way to end:
We hope you enjoyed this series of tutorials! Remember, write to us if you want to get in touch.