Interesting links
This page is a compilation of links I found interesting while learning Python and while solving
everyday problems in project management and maintenance, and of course scientific computing. As I keep on learning this list evolves continuously :-)
“the most dangerous thought you can have as a creative person, is that you know what you are doing.” Bret Victor - The Future of Programming.
“Every great developer you know got there by solving problems they were unqualified to solve until they actually did it.” - Patrick McKenzie
It´s better to wait for a productive programmer to become available than it is to wait for the first available programmer to become productive. - Steve McConnell
Debugging is twice as hard as writing the code in the first place.Therfore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it. - Rajanand
Always code as if the guy who ends up maintaning your code will be a violent psychopath who knows where you live. - Rick Osborne
The trouble with programmers is that you can never tell what a programmer is doing until it’s too late. - Seymour Cray
First, solve the problem. Then write the code. - John Johnson
Code is like humor. When you have to explain it, it’s bad. – Cory House
Simplicity is the soul of efficiency. – Austin Freeman
Most of you are familiar with the virtues of a programmer. There are three, of course: laziness, impatience, and hubris.
C makes it easy to shoot yourself in the foot; C++ makes it harder, but when you do, it blows your whole leg off.
A system administrator has two problems: 1. Dumb users. 2. Smart users.
“Never trust a programmer in a suit.” - Anonymous
“Fools ignore complexity. Pragmatists suffer it. Some can avoid it. Geniuses remove it.” - Alan J. Perlis
“Theory and practice sometimes clash. And when that happens, theory loses. Every single time.” - Linus Torvalds
For Beginners
De Programmeursleerling - Pieter Spronck (in Dutch)
Python as a language
Python is known for being a language that’s easy to read, quick to develop in, and applicable to a wide range of scenarios
How long did it take you to learn Python Wait, don’t answer that. It doesn’t matter. Ned Batchelder
Software engineering
What scientists must know about hardware to write fast code A simplified view - but not over-simplified - on how hardware affects performance. Written with Julia in mind rather than Python, but the principles remain valid.
Software Engineering’s Greatest Hits Very interesting.
Python internals
Python for HPC
Here’s a list of approaches that rely on low-lever programming languages, as C, C++ and Fortran, for speeding up Python (sequential) code. Some of these approaches, e.g. Numba rely on automatic code transformation from Python, so there is no need to write low-level code yourself.
Performance Python: Seven Strategies for Optimizing Your Numerical Code
How vectorization speeds up your Python code Quite a few interesting ideas: * self-instrumenting a python program for performance measurements * using pypy
Cython, Rust, and more: choosing a language for Python extensions
PyCon 22 Talk - Henry Fredrick Schreiner III: Building a binary extension
Deep CMake for library authors CppCon 2019, Interesting CMake stuff by craig scott, the author of Professional CMake - a practical guide
Approaches mimicking or wrapping OpenMP and MPI:
Pymp – OpenMP-like Python Programming A really interesting concept, not as efficient as OpenMP itself (which incurs quite a bit of overhead itself), and, of course, limited to a single node. As the number of cores per node keeps increasing, pymp may be a good solution for problems that can do with a single node.
High performance Python 4 Mpi4py, doing mpi from Python.
Other parallel processing approaches:
Sequential Execution, Multiprocessing, and Multithreading IO-Bound Tasks in Python
Visualize multi-threaded Python programs with an open source tool
GPU
Concepts and ideas:
Does it ever make sense to use more concurrent processes than processor cores? You can have as many threads as you want as long as they’re doing nothing.
Code modernization
Profiling
Memory profiling
Resource monitoring
Python idioms and readability
Practical decorators Reuven Lerner
Passing Exceptions 101 Paradigms in Error Handling - PyCon 2017
Modern Python Dictionaries: A confluence of a dozen great ideas - PyCon 2017
When Python Practices Go Wrong About the use of exec() and eval(). A presentation, so, the logic isn`t always obvious, but definitely an interesting topic. Here’s the corresponding video When Python Practices Go Wrong - Brandon Rhodes - code::dive 2019
Learn something new about Python every day in less than 1 minute
The unreasonable effectiveness of f-strings and re.VERBOSE how to construct readable and documented regular expressions.
Useful packages
Exceptions
Type checking in Python
Design patterns
Testing
tox nox and invoke Break the Cycle: Three excellent Python tools to automate repetitive tasks
Beyond Unit Tests: Taking Your Testing to the Next Level - PyCon 2018
“WHAT IS THIS MESS?” - Writing tests for pre-existing code bases - PyCon 2018
Property-based tests for the Python standard library (and builtins)
The Clean Architecture in Python - How to write testable and flexible code
15 amazing pytest plugins and more (an episode on an interesting blog).
Property-Based Testing with hypothesis, and associated use cases
Learning Python Test Automation These days, there’s a wealth of great content on Python testing. Here’s a brief reference to help you get started.
Debugging
Liran Haimovitch - Understanding Python’s Debugging Internals - PyCon 2019
When C extensions crash: easier debugging for your Python application
- All about gdb, TUI and python inside gdb:
Logging
Scientific Python
A Gentle Introduction to Serialization for Python (pickle, hdf5)
Matplotlib Tutorial (2022): For Physicists, Engineers, and Mathematicians
SciPy Tutorial (2022): For Physicists, Engineers, and Mathematicians
NumPy Tutorial (2022): For Physicists, Engineers, and Mathematicians
SymPy Tutorial (2022): For Physicists, Engineers, and Mathematicians
Artificial intelligence, machine learning and data science
Scikit-learn, wrapping your head around machine learning - PyCon 2019
Thinking like a Panda: Everything you need to know to use pandas the right way
streamz: Build Pipelines to Manage Continuous Streams of Data
A series how to turn machine learning models into production-ready software solutions
µTransfer: A technique for hyperparameter tuning of enormous neural networks
Pandas
CLIs and scripting
GUI
Packaging
Inside the Cheeseshop: How Python Packaging Works - PyCon 2018 historical overview with thorough explanation
Share Your Code! Python Packaging Without Complication - PyCon 2017
Options for packaging your Python code: Wheels, Conda, Docker, and more
How to improve Python packaging, or why fourteen tools are at least twelve too many
Graphics
Installing packages
Tools
IPython and Jupyter in Depth: High productivity, interactive Python Matthias Bussonier
Life Is Better Painted Black, or: How to Stop Worrying and Embrace Auto-Formatting
Using GitHub, Travis CI, and Python to Introduce Collaborative Software Development - PyCon 2018
Understanding Best Practice Python Tooling by Comparing Popular Project Templates
Leverage Sublime project folders to eashttps://martinheinz.dev/blog/34e your work
Deep dive into how pyenv actually works by leveraging the shim design pattern
How to automatically set up a development machine with Ansible
`direnv -- unclutter your .profile <https://github.com/direnv/direnv?utm_source=tldrnewsletter.`_ direnv is an extension for your shell. It augments existing shells with a new feature that can load and unload environment variables depending on the current directory.
git and other VCS
Development environment, developement workflow
pyenv+poetry+pipx <https://jacobian.org/2019/nov/11/python-environment-2020/>
https://pypi.org/project/create-python-package/ a micc ‘light’
How to Set Up a Python Project For Automation and Collaboration
Rewriting your git history, removing files permanently - cheatsheet & guide
How to Set Environment Variables in Linux and Mac: The Missing Manual
CI/CD
Problem solving
Documentation
Django
Fortran/C/C++ Syntax
C++
Johnny’s software lab very thorough site on C++ performance issues.
Compilers
Notebooks
Containers
Windows
Using WSL to Build a Python Development Environment on Windows This is promising: maybe we finally have a an environment on Windows with a minimal difference from Linux an MacOSX.
Linux
Programming blogs
QUOTES
“The code you write makes you a programmer. The code you delete makes you a good one. The code you don’t have to write makes you a great one.” - Mario Fusco
“It’s hard enough to find an error in your code when you’re looking for it; it’s even harder when you’ve assumed your code is error-free.” - Steve McConnell