nixpkgs/pkgs/development/interpreters
adisbladis d88a7735d2
Python: introduce NIX_PYTHONPREFIX in order to set site.PREFIXES
This is needed in case of `python.buildEnv` to make sure site.PREFIXES
does not only point to the unwrapped executable prefix.

--------------------------------------------------------------------------------

This PR is a story where your valiant hero sets out on a very simple adventure but ends up having to slay dragons, starts questioning his own sanity and finally manages to gain enough knowledge to slay the evil dragon and finally win the proverbial price.

It all started out on sunny spring day with trying to tackle the Nixops plugin infrastructure and make that nice enough to work with.

Our story begins in the shanty town of [NixOps-AWS](https://github.com/nixos/nixops-aws) where [mypy](http://mypy-lang.org/) type checking has not yet been seen.

As our deuteragonist (@grahamc) has made great strides in the capital city of [NixOps](https://github.com/nixos/nixops) our hero wanted to bring this out into the land and let the people rejoice in reliability and a wonderful development experience.

The plugin work itself was straight forward and our hero quickly slayed the first small dragon, at this point things felt good and our hero thought he was going to reach the town of NixOps-AWS very quickly.

But alas! Mypy did not want to go, it said:
`Cannot find implementation or library stub for module named 'nixops'`

Our hero felt a small sliver of life escape from his body. Things were not going to be so easy.

After some frustration our hero discovered there was a [rule of the land of Python](https://www.python.org/dev/peps/pep-0561/) that governed the import of types into the kingdom, more specificaly a very special document (file) called `py.typed`.
Things were looking good.

But no, what the law said did not seem to match reality. How could things be so?

After some frustrating debugging our valiant hero thought to himself "Hmm, I wonder if this is simply a Nix idiosyncrasy", and it turns out indeed it was.
Things that were working in the blessed way of the land of Python (inside a `virtualenv`) were not working the way they were from his home town of Nix (`nix-shell` + `python.withPackages`).

After even more frustrating attempts at reading the mypy documentation and trying to understand how things were supposed to work our hero started questioning his sanity.
This is where things started to get truly interesting.

Our hero started to use a number of powerful weapons, both forged in the land of Python (pdb) & by the mages of UNIX (printf-style-debugging & strace).

After first trying to slay the dragon simply by `strace` and a keen eye our hero did not spot any weak points.
Time to break out a more powerful sword (`pdb`) which also did not divulge any secrets about what was wrong.

Our hero went back to the `strace` output and after a fair bit of thought and analysis a pattern started to emerge. Mypy was looking in the wrong place (i.e. not in in the environment created by `python.withPackages` but in the interpreter store path) and our princess was in another castle!

Our hero went to the pub full of old grumpy men giving out the inner workings of the open source universe (Github) and acquired a copy of Mypy.
He littered the code with print statements & break points.
After a fierce battle full of blood, sweat & tears he ended up in 20f7f2dd71/mypy/sitepkgs.py and realised that everything came down to the Python `site` module and more specifically https://docs.python.org/3.7/library/site.html#site.getsitepackages which in turn relies on https://docs.python.org/3.7/library/site.html#site.PREFIXES .

Our hero created a copy of the environment created by `python.withPackages` and manually modified it to confirm his findings, and it turned out it was indeed the case.
Our hero had damaged the dragon and it was time for a celebration.

He went out and acquired some mead which he ingested while he typed up his story and waited for the dragon to finally die (the commit caused a mass-rebuild, I had to wait for my repro).

In the end all was good in [NixOps-AWS](https://github.com/nixos/nixops-aws)-town and type checks could run. (PR for that incoming tomorrow).
2020-03-14 21:39:31 +00:00
..
acl2 treewide: remove redundant quotes 2019-08-26 21:40:19 +00:00
angelscript angelscript: 2.33.0 -> 2.34.0 2019-10-06 17:46:34 -07:00
bats treewide: name -> pname (easy cases) (#66585) 2019-08-15 13:41:18 +01:00
ceptre treewide: remove redundant rec 2019-08-28 11:07:32 +00:00
chibi
clips treewide: remove redundant rec 2019-08-28 11:07:32 +00:00
clisp clisp: Add license 2019-04-09 13:23:55 +02:00
clojure clojure 1.10.1.492 -> 1.10.1.507 plus bugfix (#79868) 2020-02-12 11:50:50 +00:00
clojurescript/lumo cleanup 2020-02-19 23:40:14 +01:00
dart dart: 2.0.0 -> 2.7.1 (stable) + 2.0.0 -> 2.8.0-dev.10.0 (dev) 2020-02-27 14:23:27 +01:00
dhall Add Nixpkgs support for Dhall 2020-02-11 22:02:53 -08:00
duktape duktape: 2.4.0 -> 2.5.0 2019-12-08 21:47:09 +01:00
eff Treewide: fix more URL permanent redirects 2019-11-21 15:37:34 -08:00
elixir elixir_1_10: 1.10.1 -> 1.10.2 2020-02-26 13:12:21 +01:00
erlang treewide: installTargets is a list 2019-12-31 00:25:26 +01:00
evcxr evcxr: upgrade cargo fetcher and cargoSha256 2020-02-15 22:09:05 -08:00
falcon treewide: remove redundant rec 2019-08-28 11:07:32 +00:00
gauche gauche: 0.9.8 -> 0.9.9 2019-12-16 20:20:20 -05:00
gnu-apl treewide: NIX_*_FLAGS -> string 2019-12-31 00:15:46 +01:00
groovy groovy: 2.5.8 -> 2.5.9 2020-01-21 15:34:14 +00:00
gtk-server gtk-server: 2.3.1 -> 2.4.5 2019-12-15 13:31:53 -08:00
guile guile: 2.2.6 -> 2.2.7 2020-03-11 23:56:38 +00:00
hugs
hy hy: use python2, build fails with 3 2019-11-13 16:27:38 +01:00
icon-lang icon-lang: fix build 2020-02-23 17:22:13 +01:00
io treewide: Get rid of libGLU_combined 2019-11-18 20:10:43 +00:00
j treewide: name -> pname (easy cases) (#66585) 2019-08-15 13:41:18 +01:00
janet janet: 1.6.0 -> 1.7.0 2020-02-07 10:54:34 +00:00
jimtcl treewide: NIX_*_FLAGS -> string 2019-12-31 00:15:46 +01:00
joker joker: 0.14.1 -> 0.14.2 2020-03-05 02:36:00 +00:00
jruby jruby: 9.2.10.0 -> 9.2.11.0 2020-03-05 02:42:20 +00:00
jython jython: 2.7.2b2 -> 2.7.2b3 2020-02-23 13:41:27 -08:00
kona treewide: replace make/build/configure/patchFlags with nix lists 2019-12-30 12:58:11 +01:00
lfe lfe: 1.2.1 -> 1.3 2020-02-10 20:03:47 +01:00
lolcode treewide: name -> pname (easy cases) (#66585) 2019-08-15 13:41:18 +01:00
love treewide: NIX_*_FLAGS -> string 2019-12-31 00:15:46 +01:00
lua-5 buidLuarocksPackage: add a checkPhase 2020-02-26 01:14:30 +01:00
luajit luajit: Expose build options, enable JIT debug module 2020-02-15 18:40:02 +01:00
lush treewide: Get rid of libGLU_combined 2019-11-18 20:10:43 +00:00
maude maude: update from version 2.7.1 to 3.0 (including full-maude) 2020-01-31 16:25:40 +01:00
metamath metamath: 0.180 -> 0.181 2020-03-14 03:24:44 +00:00
micropython micropython: init at 1.12 2020-01-03 10:57:55 +01:00
mujs treewide: name -> pname (easy cases) (#66585) 2019-08-15 13:41:18 +01:00
nix-exec treewide: name -> pname 2019-08-17 10:54:38 +00:00
octave octave: make optional features declarative in default.nix 2020-02-26 10:09:09 +02:00
perl perl: Enable threading on darwin 2020-02-20 08:35:45 +01:00
php php72: 7.2.27 -> 7.2.28 2020-03-11 20:13:56 +01:00
picoc treewide: name -> pname (easy cases) (#66585) 2019-08-15 13:41:18 +01:00
picolisp picolisp: 19.6 -> 19.12 2020-01-06 22:25:42 -08:00
pixie treewide: remove redundant quotes 2019-09-08 23:38:31 +00:00
proglodyte-wasm treewide: mark some broken packages as broken 2019-10-08 17:14:26 +02:00
pure pure: mark as broken 2020-01-30 18:35:30 -05:00
pyrex treewide: name -> pname (#67513) 2019-08-31 07:41:22 -04:00
python Python: introduce NIX_PYTHONPREFIX in order to set site.PREFIXES 2020-03-14 21:39:31 +00:00
qnial treewide: remove redundant rec 2019-08-28 11:07:32 +00:00
quickjs treewide: Remove myself from maintainers on some packages (#78027) 2020-01-19 12:18:34 -05:00
racket racket: 7.5 -> 7.6 2020-02-14 04:20:00 -05:00
rakudo rakudo: 2020.01.1 -> 2020.02 2020-02-27 22:00:27 -06:00
rascal treewide: use dontUnpack 2019-07-01 04:23:51 -04:00
rebol treewide: name -> pname (easy cases) (#66585) 2019-08-15 13:41:18 +01:00
red treewide: name -> pname (easy cases) (#66585) 2019-08-15 13:41:18 +01:00
regina treewide: name -> pname (easy cases) (#66585) 2019-08-15 13:41:18 +01:00
renpy treewide: Get rid of libGLU_combined 2019-11-18 20:10:43 +00:00
ruby ruby_2_4: remove 2020-02-10 13:23:35 -05:00
scheme48
scsh
self treewide: remove redundant rec 2019-08-28 11:07:32 +00:00
spidermonkey treewide: replace make/build/configure/patchFlags with nix lists 2019-12-30 12:58:11 +01:00
supercollider supercollider: 3.10.3 -> 3.10.4 2020-01-30 11:03:45 +00:00
tcl tcl: fix dangling symlink 2019-12-19 09:46:36 -05:00
tinyscheme treewide: name -> pname (easy cases) (#66585) 2019-08-15 13:41:18 +01:00
unicon-lang treewide: remove redundant rec 2019-08-28 11:07:32 +00:00
wasmer wasmer: 0.13.0 -> 0.16.2 2020-03-12 19:06:47 +01:00
wasmtime wasmtime: bump to v0.12.0 (from v0.8.0) 2020-03-01 02:34:43 +01:00