HOWTO write efficient python

In other languages: English | Afrikaans | Albanian | Arabic | Brazilian | Bulgarian | Catalan | Chinese | Croatian | Czech | Danish | Dutch | Esperanto | Estonian | Filipino | Finnish | Flemish | French | German | Greek | Hebrew | Hindi | Hungarian | Indonesian | Italian | Japanese | Latvian | Lithuanian | Macedonian | Malay | Malayalam | Norwegian (Bokmål) | Norwegian (Nynorsk) | Persian | Polish | Portuguese | Romanian | Russian | Serbian | Slovak | Slovenian | Spanish | Swedish | Turkish | Ukrainian | Urdu

Every code is efficient being simple, easily readable, easily maintainable and sufficiently rapid.

Quick References

Programming, as a problem solving art, is the process of knowing what to do and how to do. Quick References are made to help us in the "how to do" stuff. Thankfully, we have the Python Quick Reference.

Optimizing your Scripts

When your python script begins to become a behemoth, and you'd like to try and speed things up, they are a few quick tips on achieving decent performance gain.

Python Interpreter Optimization

Python has a command line option called -O, which performs normal optimization. There is also -OO, which doesn't have much effect, but can still give a bit more yield than -O. Try the following on your script:

python -OO -c "from py_compile import compile; compile('/home/usr/path/to/script.py')"

Let's say your script is called script.py. When the above is ran, it will generate a file called script.pyo. Instead of running something such as "./script.py" or "python script.py", do "./script.pyo". This will launch the optimized version of your script. If you still run the script from the .py file, it will not use the optimized file!

Using List Comprehensions and Writing Pythonic Code

List comprehensions are a good way to build lists, not just because they run faster, but they also look better. For example, instead of writing:

my_list = []
for x in range(100):
  my_list.append(x)

Write:

my_list = [x for x in range(100)]

List comprehensions can also be used to run a function against every element of a list. For example, instead of writing:

for item in my_list:
  some_function(item)

Write:

[some_function(item) for item in my_list]

Or, use map:

map(my_func,my_list)

Another example,

Instead of:

for i in xrange(0,len(lst),+2):
  lst[i]=False

Write:

lst[::2]=[False]*len(lst[::2])

Using Psyco

Psyco is a library for Python which compiles high-load functions in a script, making the roundabout time for using them much shorter. If you have installed Psyco, try this in a script:

import psyco
psyco.full()

That's it! Two lines, and your program should already be running considerably faster. You can also use Psyco's bind() function to choose which functions to speed up.

Using C/C++

Python is written in C/C++ and you can create new Python modules[1] with your own functions in these languages. C and C++ are faster than Python because they are compiled in native code, running directly in the processor.