Sep 6, 2018

1 min read

Speedup numerical operations using Pydata’s NumExpr module

In this story, I will explain how I knew about the numexpr module and how it can be leveraged to speed up the numerical operations in python applications.

A few days ago, I was solving a bug related to sympy.utilities.lamdify module and in its documentation I found a small paragraph about the numExpr module. So, I started exploring it.

According to its documentation:

NumExpr is a fast numerical expression evaluator for NumPy. With it, expressions that operate on arrays (like '3*a+4*b') are accelerated and use less memory than doing the same calculation in Python.

In addition, its multi-threaded capabilities can make use of all cores — which generally results in substantial performance scaling compared to NumPy.

Last but not least, numexpr can make use of Intel’s VML (Vector Math Library, normally integrated in its Math Kernel Library, or MKL). This allows further acceleration of transcendent expressions. details

In their documentation they also provided some example which can be seen here:

But I further took these example and did time comparison for operation by using numpy and numexpr:

It can be seen that Numexpr has greatly improved the performance of the numpy operation.

DevSecOps Engineer

Love podcasts or audiobooks? Learn on the go with our new app.