Dr. Mario's 2nd 🧠

High Performance Python

5 min read

rw-book-cover

Metadata

Highlights

While this book is primarily aimed at people with CPU-bound problems, we also look at data transfer and memory-bound solutions. (Location 64)

Vectorization occurs when a CPU is provided with multiple pieces of data at a time and is able to operate on all of them at once. This sort of CPU instruction is known as single instruction, multiple data (SIMD). (Location 235)

Amdahl’s law. Simply stated, Amdahl’s law is this: if a program designed to run on multiple cores has some subroutines that must run on one core, (Location 257)

frequency). It is important to note that the data moved in one transfer is always sequential: a chunk of data is read off of the memory and moved to a different place. Thus, the speed of a bus is broken into these two quantities because individually they can affect different aspects of computation: a large bus width can help vectorized code (or any code that sequentially reads through memory) by making it possible to move all the relevant data in one transfer, while, on the other hand, having a small bus width but a very high frequency of transfers can help code that must do many reads from random parts of memory. (Location 346)

This theme of keeping data where it is needed and moving it as little as possible is very important when it comes to optimization. The concept of “heavy data” refers to the time and effort required to move data around, which is something we would like to avoid. (Location 395)

This is possible because the CPU vectorizes operations with no additional time cost, meaning it can do multiple independent computations at the same time. (Location 399)

Python’s abstraction hurts any optimizations that rely on keeping the L1/L2 cache filled with the relevant data for the next computation. This comes from many factors, the first being that Python objects are not laid out in the most optimal way in memory. This is a consequence of Python being a garbage-collected language — memory is automatically allocated and freed when needed. This creates memory fragmentation (Location 451)

A second, more fundamental problem comes from Python’s dynamic types and the language not being compiled. As many C programmers have learned throughout the years, the compiler is often smarter than you are. (Location 457)

Overall team velocity is far more important than speedups and complicated solutions. (Location 537)

In a research environment, it is common to create many Jupyter Notebooks using poor coding practices while iterating through ideas and different datasets. (Location 546)

Make it work (Location 550)

“We saved an hour’s thinking by coding all afternoon.” In some fields this is better known as “Measure twice, cut once.” (Location 553)

  • Note: BuenĂ­simo! Es justo la contrapartida a lo que se dice en XP con lo de “en lugar de elucubrar, podrian implementar directamente”

Make it right (Location 554)

Make it fast (Location 556)

Using assert statements to check data in your code should be frowned upon. It is an easy way to assert that certain conditions are being met, but it isn’t idiomatic Python. To make your code easier to read by other developers, check your expected data state and then raise an appropriate exception if the check fails. A common exception would be ValueError if a function encounters an unexpected value. (Location 605)

  • Tags: pink
  • Note: Tenemos nosotros esto mismo en el cĂłdigo

The Bulwark library (Location 608)

You may also want to add some sanity checks at the end of your Notebook — a mixture of logic checks and raise and print statements that demonstrate that you’ve just generated exactly what you needed. (Location 609)

The overhead of using this decorator is small, but if you’re calling fn millions of times, the overhead might become noticeable. (Location 854)

The timeit module temporarily disables the garbage collector. This might impact the speed you’ll see with real-world operations if the garbage collector would normally be invoked by your operations. See the Python documentation for help on this. (Location 879)

Probably the most useful indicator here is Major (requiring I/O) page faults, as this indicates whether the operating system is having to load pages of data from the disk because the data no longer resides in RAM. This will cause a speed penalty. (Location 938)

is worthwhile printing and annotating versions of the output from this tool as you modify your code, so you have a record of changes (successful or not) that you can quickly refer to. Don’t rely on your memory when you’re working on line-by-line changes. (Location 1200)

We should reject the hypothesis that in Python 3.7 changing the order of the logic results in a consistent speedup — there’s no clear evidence for this. Ian (Location 1304)

Generally, it is more efficient to overallocate memory in a process that can be used at leisure, as memory allocation operations are relatively expensive. Furthermore, garbage collection is not instantaneous, so objects may be unavailable but still in the garbage collection pool for some time. (Location 1376)

these buckets can store any type of data we want (as opposed to numpy arrays, which have a static type and can store only that type of data).2 (Location 1787)

For tuples of sizes 1–20, however, when they are no longer in use, the space isn’t immediately given back to the system: up to 20,000 of each size are saved for future use. (Location 2049)

This is yet another reason to be explicit about what functions you are importing from a module. This practice not only makes code more readable, because the reader knows exactly what functionality is required from external sources, but it also simplifies changing the implementation of specific functions and generally speeds up code! (Location 2503)

Webmentions

Loading webmentions...

Graph View