- Properly manage and lock versions of `nixpkgs` and `mach-nix` for environments created via `mach-nix env` command.
- Add example on how to use mach-nix with jupyterWith
### Improvements
- Improve portability of `mach-nix env` generated environments. Replace the platform specific compiled nix expression with a call to mach-nix itself, which is platform agnostic.
- Mach-nix now produces the same result no matter if it is used through flakes or legacy interface. The legacy interface now loads its dependencies via `flakes.lock`.
### Fixes
- mkDockerImage produced corrupt images.
- non-python packages passed via `packagesExtra` were not available during runtime. Now they are added to the `PATH`.
- remove `<nixpkgs>` impurity in the dependency extractor used in buildPythonPackage.
- Some sdist packages were missing from the dependency DB due to a corrupt index in the SQL DB used by the crawler.
- When automatically fixing circular deps, removed deps could trigger a `No matching distribution found` error in higher level parent packages. Now `--no-dependencies` is set recursively for all parents of removed deps.
- Mapping out the resulting dependency DAG to a tree for printing could exhaust the systems resources, due to complexity. Now, when printing dependencies, sub-trees are trimmed and marked via (...) if they have already been printed earlier.
### Improvements
- optimized autoPatchelfHook for faster processing of large wheel packages (see [upstream PR](https://github.com/NixOS/nixpkgs/pull/101142))
-`networkx` is now used for dealing with some graph related problems
flakes pypi gateway, R support, new output formats, more packages for python 3.5/3.6, improved providers nixpkgs/wheel
### IMPORTANT NOTICE
The UI has been reworked. It is backward compatible with a few exceptions. Most importantly, when importing mach-nix, an attribute set must be passed. It can be empty. Example:
- R support (experimental): R packages can be passed via `packagesExtra`. Mach-nix will setup rpy2 accordingly. See [usage example](https://github.com/DavHau/mach-nix/blob/master/examples.md#r-and-python).
- Non-python packages can be passed via `packagesExtra` to include them into the environment.
### Improvements
- rework the logic for inheriting dependencies from nixpkgs
- fixes.nix: allow alternative mod function signature with more arguments:
Global conditional overrides, simple overrides for buildPythonPackage, improved metadata extraction, fix wheel selection
### Features
- **Global conditional overrides**: Similar to the overrides from poetry2nix, this allows users to upstream their 'fixes' for python packages. Though, a special format is used here which is optimized for human readability and allows to define a condition for each fix. Therefore fixes are applied on a granular basis depending on the metadata of each package like its `version`, `python version`, or `provider`. This format is designed in a way, so it could easily be reused by projects other than mach-nix. Please contribute your fixes to [./mach_nix/fixes.nix](https://github.com/DavHau/mach-nix/blob/master/mach_nix/fixes.nix)
- Simplified overrides are now also available for buildPythonPackage (underscore argument)
- Inherit passthru from nixpkgs: Reduces risk of missing attributes like `numpy.blas`.
- Allow passing a string to the `python` argument of mkPython: Values like, for example, `"python38"` are now accepted in which case `pkgs.python38` will be used. The intention is to reduce the risk of accidentally mixing multiple nixpkgs versions.
- Improved error handling while extracting metadata from python sources in buildPythonPackage.
### Fixes
- Selecting `extras` when using `buildPythonPackage` didn't have any effect
- The `passthru` argument for `buildPythonPackage` was ignored
- The `propagatedBuildInputs` argument for `buildPythonPackage` was ignored
- Wheels with multiple python versions in their filename like `PyQt5-...-cp35.cp36.cp37.cp38-...whl` were not selected correctly.
### Package Fixes:
- tensorflow: collision related to tensorboard
- orange3: broken .so file caused by fixupPhase (probably due to shrinking)
- Improved selection of wheel releases. MacOS is now supported and architectures besides x86_64 should be handled correctly.
- Whenever mach-nix resolves dependencies, a visualization of the resulting dependency tree is printed on the terminal.
- The dependency DB is now accessed through a caching layer which reduces the resolver's CPU time significantly for larger environments.
- The python platform context is now generated from the nix build environment variable `system`. This should decrease the chance of impurities during dependency resolution.
### Fixes
- The requires_python attribute of wheels was not respected. This lead to failing builds especially for older python versions. Now `requires_python` is part of the dependency graph and affects resolution.
- Detecting the correct package name for python packages in nixpkgs often failed since the attribute names don't follow a fixed schema. This lead to a handful of different errors in different situations. Now the package names are extracted from the pypi `url` inside the `src` attribute which is much more reliable. For packages which are not fetched from pypi, the `pname` attribute is used as fallback.
- Fixed bug which lead to the error `attribute 'sdist' missing` if a package from the nixpkgs provider was used which doesn't publish it's source on pypi. (For example `tensorflow`)
### Other Changes
- Mach-nix now uses a revision of the nixpkgs-unstable branch instead of nixos-20.03 as base fo the tool and the nixpkgs provider.
- Some wheel packages could brake through patchelf if they already contained stripped binaries. Packages like numpy wouldn't work because of this. This is now fixed by passing `dontStrip` to the `autoPatchelf` routine.
Improves build-time closure, build success rate and fixes disable_checks option.
### Fixes:
- fix: `disable_checks` did not work for packages built via the `sdist` provider if the required version matches exactly the version used in nixpkgs.
- fix: some dependencies with markers were ignored completely
- fix: providers `nixpkgs` and `sdist` inherited many unneeded build inputs from nixpkgs leading to bloated build-time closures, increased failure rate and uneffective `disable_checks` option. After this fix, only non-python build-inputs are inherited from nixpkgs.
- mach-nix now sets `pname` + `version` for python packages instead of `name`
- **Python Wheels**: Wheels are supported for linux including manylinux1, manylinux2010, manylinux2014
- **Nixpkgs python packages**: Nixpkgs is now treated as a separate provider. That means packages can be taken directly from there, independent on their availability from pypi.
- **Configurable Providers**: The user can now freely decide where packages should be taken from either via a default or on a per package basis. Currently available provider are `nixpkgs`, `sdist`, `wheel`.
- **Core architecture - Python Overrides**: The core of mach nix now produces python overrides instead of nixpkgs overlays which makes it much more composible and allows for a less strange API.
- **Overlay support**: `mkPython` and lower level interfaces now accept new parameter `pkgs` and `python` which indirectly enables you to use any nixpkgs overlays (But don't use this for modifying python packages. Better do this with **Python Overrides**)
- **Python Overrides support**: `mkPython` now accepts new parameter `overrides_pre` and `overrides_post` each allowing to inject a list of python overrides to apply before and after the mach-nix internal overrides. This allows you to fixup packages or add your own modifications to them.
- **Extras support**: All providers now fully support python [extras](https://www.python.org/dev/peps/pep-0508/#extras). That means requirements like for example '`requests[security]`' will be correctly resolved.
