5.2. Packaging ML projects

5.2.1. Generate requirements.txt file automatically from notebooks

Jupyter notebooks are jungles of monstrosity under the hood.

So, IDEs like PyCharm or VS Code can’t automatically generate accurate requirements.txt files because they only work on scripts.

You can easily solve this by using the ‘pipreqsnb’ library which generates dependency information from both notebooks and scripts.

Add –force tag to outwrite existing requirements files.

5.2.2. pipdeptree for much better dependency management

Raise your hand if you used “pip freeze” and vowed to yourself you will never, ever use it again!

I handle dependency conflicts at least once a week - the process is still a mess in Python. Fortunately, I have recently come across a tool called “pipdeptree” which allows you to see dependencies of your environment in a hierarchical fashion.

The library also gives you warnings when there are version conflicts or even worse, circular dependencies (that’s usually the sign you have to delete the whole conda environment).

Repo: https://github.com/naiquevin/pipdeptree