urlopen#

class urlopen(url, filename=None, save=None, headers=None, params=None, data=None, prefix='http', convert=True, die=False, return_response=False, verbose=False)[source]#

Download a single URL.

Alias to urllib.request.urlopen(url).read(). See also sc.download() for downloading multiple URLs. Note: sc.urlopen()/sc.wget() are aliases.

Parameters:

  • url (str) – the URL to open, either as GET or POST

  • filename (str) – if supplied, save to file instead of returning output

  • save (bool) – if supplied instead of filename, then use the default filename

  • headers (dict) – a dictionary of headers to pass

  • params (dict) – a dictionary of parameters to pass to the GET request

  • data (dict) –

  • prefix (str) – the string to ensure the URL starts with (else, add it)

  • convert (bool) – whether to convert from bytes to string

  • die (bool) – whether to raise an exception if converting to text failed

  • return_response (bool) – whether to return the response object instead of the output

  • verbose (bool) – whether to print progress

Examples:

html = sc.urlopen('sciris.org') # Retrieve into variable html
sc.urlopen('http://sciris.org', filename='sciris.html') # Save to file sciris.html
sc.urlopen('http://sciris.org', save=True, headers={'User-Agent':'Custom agent'}) # Save to the default filename (here, sciris.org), with headers
New in version 2.0.0: renamed from wget to urlopen; new arguments
New in version 2.0.1: creates folders by default if they do not exist
New in version 2.0.4: “prefix” argument, e.g. prepend “http://” if not present