Python etc
6.12K subscribers
18 photos
194 links
Regular tips about Python and programming in general

Owner — @pushtaev

© CC BY-SA 4.0 — mention if repost
Download Telegram
If you want to assert that a function returns a sequence with exactly one item, you can assign the result to the one-item sequence. The syntax may look a little weird:

In [1]: def echo(*args):
...: return args
...:
In [2]: a, = echo(1)
In [3]: a
Out[3]: 1
In [4]: a, = echo(1, 2)
...
ValueError: too many values to unpack (expected 1)

That solution automatically unpacks that single element to the variable which may or may not be desirable.
It looks like sum([a, b, c]) is equivalent for a + b +c, while in fact it's 0 + a + b + c. That means that it can't work with types that don't support adding to 0:

class MyInt:
def __init__(self, value):
self.value = value
def __add__(self, other):
return type(self)(self.value + other.value)
def __radd__(self, other):
return self + other
def __repr__(self):
class_name = type(self).__name__
return f'{class_name}({self.value})'

In : sum([MyInt(1), MyInt(2)])
...
AttributeError: 'int' object has no attribute 'value'

To fix that you can provide custom start element that is used instead of 0:

In : sum([MyInt(1), MyInt(2)], MyInt(0))
Out: MyInt(3)

sum is well-optimized for summation of float and int types but can handle any other custom type. However, it refuses to sum bytes, bytearray and str since join is well-optimized for this operation:

In : sum(['a', 'b'], '')
...
TypeError: sum() can't sum strings [use ''.join(seq) instead]

In : ints = [x for x in range(10_000)]
In : my_ints = [Int(x) for x in ints]
In : %timeit sum(ints)
68.3 µs ± 142 ns per loop (mean ± std. dev. of 7 runs, 10000 loops each)
In : %timeit sum(my_ints, Int(0))
5.81 ms ± 20.5 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
You can customize index completions in Jupiter notebook by providing the _ipython_key_completions_ method. This way you can control what is displayed when you press tab after something like d["x:
Note that the method doesn't get the looked up string as an argument.
Say you want to get the first N elements of an iterable. The straightforward way is to use islice:

In : from itertools import islice
In : def fib():
...: a, b = 0, 1
...: while True:
...: yield b
...: a, b = b, (a + b)
...:
In : list(islice(fib(), 5))
Out: [1, 1, 2, 3, 5]

If you also want to have indices for elements you can also apply enumerate:

In : list(enumerate(islice(fib(), 5)))
Out: [(0, 1), (1, 1), (2, 2), (3, 3), (4, 5)]

Another way to do this is to use zip and range which may seem more readable for you:

In : list(zip(range(5), fib()))
Out: [(0, 1), (1, 1), (2, 2), (3, 3), (4, 5)]
collections.defaultdict allows you to create a dictionary that returns the default value if the requested key is missing (instead of raising KeyError). To create a defaultdict you should provide not a default value but a factory of such values.

That allows you to create a dictionary that virtually contains infinite levels of nested dictionaries, allowing you to do something like d[a][b][c]...[z].

>>> def infinite_dict():
... return defaultdict(infinite_dict)
...
>>> d = infinite_dict()
>>> d[1][2][3][4] = 10
>>> dict(d[1][2][3][5])
{}

Such behavior is called “autovivification”, the term came from the Perl language.
If a function argument has the default value of None and is annotated as T, mypy automatically treats it as Optional[T] (in other words, Union[T, None]).

That doesn't work with other types, so you can't have something like f(x: A = B()). It also doesn't work with a variable assignment: a: A = B() will cause an error.

def f(x: int = None):
reveal_type(x)

def g(y: int = 'x'):
reveal_type(y)

z: int = None
reveal_type(z)

$ mypy test.py
test.py:2: error: Revealed type is 'Union[builtins.int, None]'
test.py:4: error: Incompatible default for argument "y" (default has type "str", argument has type "int")
test.py:5: error: Revealed type is 'builtins.int'
test.py:7: error: Incompatible types in assignment (expression has type "None", variable has type "int")
test.py:8: error: Revealed type is 'builtins.int'
Some Python modules are compiled into the interpreter itself. They are called built-in modules, not to be confused with the standard library. One can use sys.builtin_module_names to get the full list of such modules. The notable examples are sys, gc, time and so on.

Usually you don't care whether the module is built-in or not; however, you should be aware, that import always looks for a module among built-ins first. So, the built-in sys module is loaded even if you have sys.py available. On the other hand, if you have, say, datetime.py in the current directory it indeed can be loaded instead of the standard datetime module.
Reduce is a higher-order function that processes an iterable recursively, applying some operation to the next element of the iterable and the already calculated value. You also may know it termed fold, inject, accumulate or somehow else.

Reduce with result = result + element brings you the sum of all elements, result = min(result, element) gives you the minimum and result = element works for getting the last element of a sequence.

Python provides the reduce function (that was moved to functions.reduce in Python 3):

In : reduce(lambda s, i: s + i, range(10))
Out: 45
In : reduce(lambda s, i: min(s, i), range(10))
Out: 0
In : reduce(lambda s, i: i, range(10))
Out: 9

Also, if you ever need such simple lambdas like a, b: a + b, Python got you covered with the operator module:

In : from operator import add
In : reduce(add, range(10))
Out: 45
The biggest drawback of objects with __slots__ is that they can't dynamically have arbitrary attributes. However, you can mix the __slots__ approach with the regular __dict__ one.

To enable dynamic assignment for the object just put '__dict__' into __slots__:

class A:
__slots__ = ('a', 'b', '__dict__')

A().x = 3


Also, mind that inherited classes automatically has __dict__ unless empty __slots__ is explicitly specified:

class A:
__slots__ = ('a', 'b')

class B(A):
pass

B().x = 3
In Python 3, once the except block is exited, the variables that store caught exceptions are removed from locals() even if they previously existed:

>>> e = 2
>>> try:
... 1/0
... except Exception as e:
... pass
...
>>> e
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'e' is not defined

If you want to save a reference to the exception, you have to use another variable:

>>> error = None
>>> try:
... 1/0
... except Exception as e:
... error = e
...
>>> error
ZeroDivisionError('division by zero',)

This is not true for Python 2.
You may have your own pypi repository. It lets you release packages inside your project and install them with pip as though they are regular packages.

It is remarkable that you don’t have to install any specific software, but can use a regular http-server instead. Here is how it works for me, for example.

Let’s have trivial pacakge named pythonetc.

setup.py:

from setuptools import setup, find_packages

setup(
name='pythonetc',
version='1.0',
packages=find_packages(),
)

pythonetc.py:

def ping():
return 'pong'

Let’s release it to the ~/pypi directory:

$ python setup.py sdist bdist_wheel

$ mv dist ~/pypi/pythonetc

Now server this on the pypi.pushtaev.ru domain with nginx:

$ cat /etc/nginx/sites-enabled/pypi
server {
listen 80;
server_name pypi.pushtaev.ru;
root /home/vadim/pypi;

index index.html index.htm index.nginx-debian.html;

location / {
autoindex on;
try_files $uri $uri/ =404;
}
}

It now can be installed:

$ pip install -i http://pypi.pushtaev.ru --trusted-host pypi.pushtaev.ru pythonetc

Collecting pythonetc
Downloading http://pypi.pushtaev.ru/pythonetc/pythonetc-1.0-py3-none-any.whl
Installing collected packages: pythonetc
Successfully installed pythonetc-1.0
$ python
Python 3.7.0+ (heads/3.7:0964aac, Mar 29 2019, 00:40:55)
[GCC 4.9.2] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pythonetc
>>> pythonetc.ping()
'pong'
If you want to create a dictionary from the know set of keys and some fixed value for all of them you can you use dictionary comprehensions:

>>> keys = ['a', 'b', 'c']
>>> {k: True for k in keys}
{'a': True, 'b': True, 'c': True}

However, the dict class has the fromkeys classmethod designed specially for this case:

>>> dict.fromkeys(keys, True)
{'a': True, 'b': True, 'c': True}
Ads time!
@pythonbooks — the channel with English and Russian Python books for those of you who wants to become True Python Developer. Download books here @pythonbooks.
The map function calls another function for every element of some iterable. That means that function should accept a single value as an argument:

In : list(map(lambda x: x ** 2, [1, 2, 3]))
Out: [1, 4, 9]

However, if each element of the iterable is tuple, then it would be nice to pass each element of that tuple as a separate argument. It was possible in Python 2, thanks to the tuple parameter unpacking (note the parentheses):

>>> map(lambda (a, b): a + b, [(1, 2), (3, 4)])
[3, 7]


In Python 3, this feature is gone, but there is another solution. itertools.starmap unpacks tuple for you, as though a function is called with a star: f(*arg) (hence the function's name):

In [3]: list(starmap(lambda a, b: a + b, [(1, 2), (3, 4)]))
Out[3]: [3, 7]
It’s quite often when you have to declare a dictionary with all keys equal to the local variables with the same name. Something like this:

dict(
context=context,
mode=mode,
action_type=action_type,
)

ECMAScript even has the special form of object literal for such cases (it’s called Object Literal Property Value Shorthand):

> var a = 1;
< undefined
> var b = 2;
< undefined
> {a, b}
< {a: 1, b: 2}

It is possible to create a similar helper in Python (alas, it looks not even closely as good as the ECMAScript notation):

def shorthand_dict(lcls, names):
return {k: lcls[k] for k in names}

context = dict(user_id=42, user_ip='1.2.3.4')
mode = 'force'
action_type = 7

shorthand_dict(locals(), [
'context',
'mode',
'action_type',
])
You may wonder why we have to pass locals() as a parameter in the previous example. Is it possible to get the locals of the caller in the callee? It is indeed, but you have to mess with the inpsect module:

import inspect

def shorthand_dict(names):
lcls = inspect.currentframe().f_back.f_locals
return {k: lcls[k] for k in names}

context = dict(user_id=42, user_ip='1.2.3.4')
mode = 'force'
action_type = 7

shorthand_dict([
'context',
'mode',
'action_type',
])

You can go even further and use something like this — https://github.com/alexmojaki/sorcery:

from sorcery import dict_of
dict_of(context, mode, action_type)
Any JSON is syntaxicaly correct Python code. However, true, false and null are not defined by default. Defining them makes it possible to use eval as a JSON parser (which isn’t a good idea anyway):

$ cat json
{"$id":"1","currentDateTime":"2019-04-25T14:16Z","utcOffset":"00:00:00","isDayLightSavingsTime":false,"dayOfTheWeek":"Thursday","timeZoneName":"UTC","currentFileTime":132006753872039629,"ordinalDate":"2019-115","serviceResponse":null}

>>> null = None
>>> true = True
>>> false = False
>>> with open('json') as f:
... j = eval(f.read())
...
>>> j
{'currentFileTime': 132006753872039629, 'isDayLightSavingsTime': False, 'dayOfTheWeek': 'Thursday', 'utcOffset': '00:00:00', 'serviceResponse': None, '$id': '1', 'timeZoneName': 'UTC', 'ordinalDate': '2019-115', 'currentDateTime': '2019-04-25T14:16Z'}