Commit graph

641 commits

Author SHA1 Message Date
Matthew Bauer 93834fe194
Merge pull request #47230 from bhipple/fix/licenses
Remove dead code from stdenv check-meta license logic
2018-10-05 22:58:27 -05:00
aszlig b25b6e0c75
stdenv: Improve ELF detection for isELF
The isELF function only checks whether ELF is contained within the first
4 bytes of the file, which is a bit fuzzy and will also return
successful if it's a text file starting with ELF, for example:

  ELF headers
  -----------

  Some text here about ELF headers...

So instead, we're now doing a precise match on \x7fELF.

Signed-off-by: aszlig <aszlig@nix.build>
Acked-by: @Ericson2314
Closes: https://github.com/NixOS/nixpkgs/pull/47244
2018-09-25 06:55:18 +02:00
Benjamin Hipple 0b9d9ab256 Remove dead code from stdenv check-meta license logic
The `unfree` and `unfreeRedistributable` licenses both have `free = false`,
which will trigger the first portion of logic. This removes dead code to
simplify the logic.

As a follow-up, I plan to add an attribute `redistributable = [true|false]`,
which can be used by Hydra to determine whether a given package with a given
license can be included in the channel.
2018-09-23 12:48:02 -04:00
John Ericson 7319013ea1 Merge remote-tracking branch 'upstream/master' into staging 2018-09-18 16:55:42 -04:00
John Ericson 35378f0141
Merge pull request #46857 from obsidiansystems/darwin-to-linux-prep
misc pkgs: various cross fixes in preparation for darwin->linux
2018-09-18 16:52:30 -04:00
John Ericson 2111e7b742 mkDerivation: Make separateDebugInfo assertion lazier to match other assertions
This is needed to access attributes of derivations on platforms where
they cannot be built.
2018-09-18 16:25:19 -04:00
Graham Christensen b80c9ce4a9
stdenv: Validate meta.outputsToInstall
If meta.outputsToInstall is set to include absent outputs, various
tools break including channel updates and nix-env.

    grahamc@Morbo> nix-env -i -f . -A elf-header-real
    installing 'elf-header'
    error: this derivation has bad 'meta.outputsToInstall'

This patch verifies each value in meta.outputsToInstall is a valid
output. It validates this condition only if checkMeta is true.

    grahamc@Morbo> nix-build . -A elf-header-real
    error: Package ‘elf-header’ in /home/grahamc/projects/nixpkgs/pkgs/development/libraries/elf-header/default.nix:36 has invalid meta.outputsToInstall, refusing to evaluate.

    The package elf-header has set meta.outputsToInstall to: bin

    however elf-header only has the outputs: out

    and is missing the following ouputs:

      - bin

    (use '--show-trace' to show detailed location information)

Note, now the nix-env experience is decidedly worse for users who have
checkMeta set to true:

    grahamc@Morbo> nix-env -i -f . -A elf-header-real; echo $?
    0

though since this is already an issue for unfree, broken, unsupported,
and insecure validity problems I'm not sure we should do something
different here.
2018-09-18 10:38:44 -04:00
Matthew Bauer ba5717a6f5 stdenv: fix HOST_PATH change
a4630c65ca was incorrect in assuming $SHELL would be a path to the
bash derivation. In fact $SHELL will be a path to the bash executable.

Unfortunately this did not fix the original issue. So instead, we just
have to reuse initialPath can be added like PATH is.

Sorry for the inconvenience! I hadn’t thought through the effects of
the last commit.

/cc @copumpkin @ericson2314
2018-09-17 14:18:06 -05:00
Matthew Bauer a4630c65ca stdenv: add shell to HOST_PATH for backwards compatibility
To avoid breaking things, we need to make sure SHELL goes into
HOST_PATH. This reflects my changes to patch-shebangs to make it cross
compilation ready. When a script is patched from the Nix store it now
looks to HOST_PATH to get the targeted machine’s executables.
Unfortunately, this only works in native builds.
2018-09-16 15:58:21 -05:00
Jan Malakhovski b7bd0561be Merge branch 'master' into staging 2018-09-08 22:08:32 +00:00
John Ericson 773233ca77 top-level, stdenv: Make system and stdenv.system describe the hostPlatform.
Intuitively, one cares mainly about the host platform: Platforms differ
in meaningful ways but compilation is morally a pure process and
probably doesn't care, or those difference are already abstracted away.
@Dezgeg also empirically confirmed that > 95% of checks are indeed of
the host platform.

Yet these attributes in the old cross infrastructure were defined to be
the build platform, for expediency. And this was never before changed.
(For native builds build and host coincide, so it isn't clear what the
intention was.)

Fixing this doesn't affect native builds, since again they coincide. It
also doesn't affect cross builds of anything in Nixpkgs, as these are no
longer used. It could affect external cross builds, but I deem that
unlikely as anyone thinking about cross would use more explicit
attributes for clarity, all the more so because the rarity of inspecting
the build platform.
2018-09-06 08:33:51 -04:00
John Ericson 51907d257c stdenv, neovim: Use lib.warn for deprecation warnings 2018-09-05 11:40:29 -04:00
Jan Malakhovski c63ca0a431 stdenv: implement enableParallelChecking option
Works similarly to `enableParallelBuilding`, but is set by default when
`enableParallelBuilding` is set. In my experience most packages that build
fine in parallel also check fine in parallel.
2018-09-05 01:14:39 +00:00
John Ericson 06cd7c15a3 mkDerivation: Fix cross compilation
Derivations where drawing their `system` attribute from `hostPlatform`
instead of `buildPlatform`. Fix that, and add an explanatory commment.

Fixes #45993
2018-09-03 23:18:24 -04:00
John Ericson 82110ae656 stdenv: Better message for deprecated isArm
The message should say what to do instead.
2018-08-31 16:01:58 -04:00
John Ericson 2c2f1e37d4 reewide: Purge all uses stdenv.system and top-level system
It is deprecated and will be removed after 18.09.
2018-08-30 17:20:32 -04:00
Jörg Thalheim 9efffe0135 hurd: cleanup unmaintained target
This has been not touched in 6 years. Let's remove it to cause less
problems when adding new cross-compiling infrastructure.
This also simplify gcc significantly.
2018-08-28 22:18:02 +01:00
Jan Malakhovski eb5b313b16 stdenv: shadow outputs 2018-08-05 19:24:46 +00:00
Jan Malakhovski a53504f4a4 stdenv: shadow doCheck and doInstallCheck 2018-08-05 19:24:32 +00:00
John Ericson df5d1673ea
Merge pull request #44367 from obsidiansystems/no-stdenv-is-cross
treewide: Remove stdenv.isCross
2018-08-02 16:36:01 -04:00
John Ericson 34da7e2ce2 treewide: Remove stdenv.isCross
I *want* cross-specific overrides to be verbose, so I rather not have
this shorthand. This makes the syntactic overhead more proportional to
the maintainence cost. Hopefully this pushes people towards fewer
conditionals and more abstractions.
2018-08-02 15:01:58 -04:00
James Deikun bd63de114e stdenv/build-support: support .tbz and .txz tarballs 2018-07-30 15:30:16 -04:00
Frederik Rietdijk f59e6f4037 Merge master into staging 2018-07-26 16:18:53 +02:00
John Ericson 87b4a5d6a7 stdenv: Remove crossAttrs, now that it is no longer used
Fix #33302
2018-07-24 20:01:43 -04:00
Frederik Rietdijk 099c13da1b Merge staging-next into master (#44009)
* substitute(): --subst-var was silently coercing to "" if the variable does not exist.

* libffi: simplify using `checkInputs`

* pythonPackges.hypothesis, pythonPackages.pytest: simpify dependency cycle fix

* utillinux: 2.32 -> 2.32.1

https://lkml.org/lkml/2018/7/16/532

* busybox: 1.29.0 -> 1.29.1

* bind: 9.12.1-P2 -> 9.12.2

https://ftp.isc.org/isc/bind9/9.12.2/RELEASE-NOTES-bind-9.12.2.html

* curl: 7.60.0 -> 7.61.0

* gvfs: make tests run, but disable

* ilmbase: disable tests on i686. Spooky!

* mdds: fix tests

* git: disable checks as tests are run in installcheck

* ruby: disable tests

* libcommuni: disable checks as tests are run in installcheck

* librdf: make tests run, but disable

* neon, neon_0_29: make tests run, but disable

* pciutils: 3.6.0 -> 3.6.1

Semi-automatic update generated by https://github.com/ryantm/nixpkgs-update tools. This update was made based on information from https://repology.org/metapackage/pciutils/versions.

* mesa: more include fixes

mostly from void-linux (thanks!)

* npth: 1.5 -> 1.6

minor bump

* boost167: Add lockfree next_prior patch

* stdenv: cleanup darwin bootstrapping

Also gets rid of the full python and some of it's dependencies in the
stdenv build closure.

* Revert "pciutils: use standardized equivalent for canonicalize_file_name"

This reverts commit f8db20fb3a.
Patching should no longer be needed with 3.6.1.

* binutils-wrapper: Try to avoid adding unnecessary -L flags

(cherry picked from commit f3758258b8895508475caf83e92bfb236a27ceb9)
Signed-off-by: Domen Kožar <domen@dev.si>

* libffi: don't check on darwin

libffi usages in stdenv broken darwin. We need to disable doCheck for that case.

* "rm $out/share/icons/hicolor/icon-theme.cache" -> hicolor-icon-theme setup-hook

* python.pkgs.pytest: setupHook to prevent creation of .pytest-cache folder, fixes #40273

When `py.test` was run with a folder as argument, it would not only
search for tests in that folder, but also create a .pytest-cache folder.
Not only is this state we don't want, but it was also causing
collisions.

* parity-ui: fix after merge

* python.pkgs.pytest-flake8: disable test, fix build

* Revert "meson: 0.46.1 -> 0.47.0"

With meson 0.47.0 (or 0.47.1, or git)
things are very wrong re:rpath handling
resulting in at best missing libs but
even corrupt binaries :(.

When we run patchelf it masks the problem
by removing obviously busted paths.
Which is probably why this wasn't noticed immediately.

Unfortunately the binary already
has a long series of paths scribbled
in a space intended for a much smaller string;
in my testing it was something like
lengths were 67 with 300+ written to it.

I think we've reported the relevant issues upstream,
but unfortunately it appears our patches
are what introduces the overwrite/corruption
(by no longer being correct in what they assume)

This doesn't look so bad to fix but it's
not something I can spend more time on
at the moment.

--

Interestingly the overwritten string data
(because it is scribbled past the bounds)
remains in the binary and is why we're suddenly
seeing unexpected references in various builds
-- notably this is is the reason we're
seeing the "extra-utils" breakage
that entirely crippled NixOS on master
(and probably on staging before?).

Fixes #43650.

This reverts commit 305ac4dade.

(cherry picked from commit 273d68eff8f7b6cd4ebed3718e5078a0f43cb55d)
Signed-off-by: Domen Kožar <domen@dev.si>
2018-07-24 15:04:48 +01:00
John Ericson 611eefb067
Merge pull request #44025 from obsidiansystems/stdenv-deps-typo
mkDerivation: depsHostBuild -> depsHostHost
2018-07-23 16:09:21 -04:00
John Ericson ce2637c97a mkDerivation: depsHostBuild -> depsHostHost
`depsHostBuild` is not a thing, would never be a thing per the rules,
and isn't used anywhere. This is just my typo, hitherto unnoticed
because "host -> host" dependencies are by far the most obscure form.
2018-07-23 15:22:48 -04:00
Matthew Bauer cd91d1ad9f setup.sh: add HOST_PATH
HOST_PATH contains the path of the host package. This will include the
packages listed in buildInputs & depsHostHost. Use this to find
runtime commands that the host needs.

For instance to find the runtime version of perl,

$ PATH="$HOST_PATH" command -v perl
/nix/store/...-perl-5.28.0-aarch64-unknown-linux-android/bin/perl

This path should not be executed directly (it will break for cross
compilation). Only use it to find the location of executables that
will be run by your host system. Your build tools will, as always, be
available on the default PATH.
2018-07-22 23:14:49 -04:00
Matthew Bauer 37273afd27
Merge pull request #43224 from volth/patch-179
[staging] substitute(): --subst-var was silently coercing to "" if the variable does not exist.
2018-07-21 15:48:14 -04:00
Frederik Rietdijk f8701caafc Merge master into staging 2018-07-10 15:51:20 +02:00
aszlig 739c835515
stdenv-setup: Remove superfluous check for /bin/sh
The line was essentially checking whether /bin/sh exists and is
executable and if that's the case, the isScript function returns
successfully.

When asking the author of this line on IRC it seems that even they can't
remember or imagine what this was supposed to be.

In summary: Whenever /bin/sh doesn't exist during a build, *any* file
given to isScript is reported as being a script even if it isn't.

This is kinda counter-intuitive and not something what somebody would
expect from a function called "isScript".

Signed-off-by: aszlig <aszlig@nix.build>
Cc: @edolstra
2018-07-09 20:43:34 +02:00
Will Dietz 218d4dc154 make-derivation: Don't add host-suffix to fixed-output derivations names
Not only does the suffix unnecessarily reduce sharing, but it also breaks
unpacker setup hooks (e.g. that of `unzip`) which identify interesting tarballs
using the file extension.

This also means we can get rid of the splicing hacks for fetchers.
2018-07-09 11:07:10 -04:00
volth 0de0ce5893 substitute(): --subst-var was silently coercing to "" if the variable does not exist. 2018-07-08 16:32:03 +00:00
John Ericson ee9dc37e04
Merge pull request #40933 from obsidiansystems/linux-to-darwin
stdenv, binutils: Build cctools targeting macOS on Linux without pointless rebuilds
2018-05-23 11:37:43 -04:00
John Ericson 983e74ae4e stdenv: Avoid targetPlatform.isDarwin causing a mass rebuild
We want `buildPackages` to be almost the same as
`buildPackages.buildPackges`, but that is only true if most packages
don't care about the target platform. The commented code however made
them all care about whether the target platform was Darwin.
2018-05-23 10:06:08 -04:00
John Ericson 5e17335bd7 Merge remote-tracking branch 'upstream/staging' into strictDeps 2018-05-14 23:33:03 -04:00
John Ericson 330ca731e8 treewide: Get rid of all uses of crossConfig
The hack of using `crossConfig` to enforce stricter handling of
dependencies is replaced with a dedicated `strictDeps` for that purpose.
(Experience has shown that my punning was a terrible idea that made more
difficult and embarrising to teach teach.)

Now that is is clear, a few packages now use `strictDeps`, to fix
various bugs:

 - bintools-wrapper and cc-wrapper
2018-05-14 23:30:37 -04:00
Frederik Rietdijk a18b493e02 Merge master into staging 2018-05-09 10:48:01 +02:00
John Ericson 5bdfe37daf
Merge pull request #40154 from obsidiansystems/stdenv-arm-alias
stdenv: Put back isArm, with deprecation notice.
2018-05-08 14:24:58 -04:00
John Ericson 302c4c5f2d stdenv: Put back isArm, with deprecation notice.
This was always meant to be deprecated rather than removed.
2018-05-07 20:14:52 -04:00
John Ericson 4f7cdd35d5
Merge pull request #40139 from obsidiansystems/modular-setup-hooks
treewide: Modular setup hooks
2018-05-07 15:32:10 -04:00
John Ericson 34a3233a2e stdenv: Support concatenating setup hooks from multiple parts. 2018-05-07 14:43:22 -04:00
Matthew Justin Bauer eeb016e8f0
Merge branch 'staging' into fix-ncurses-darwin-extensions 2018-05-02 15:40:38 -05:00
Michael Raskin 6ca343103b check-meta.nix: specify meta.timeout 2018-04-27 00:25:37 +02:00
Jan Malakhovski 87651b32fe stdenv: steal checkInputs from buildPythonPackage
Note that a bunch of non-python packages use this attribute already.
Some of those are clearly unaware of the fact that this attribute does
not exists in stdenv because they define it but don't to add it to
their `bulidInputs` :)

Also note that I use `buildInputs` here and only handle regular
builds because python and haskell builders do it this way and I'm not
sure how to properly handle the cross-compilation case.
2018-04-26 20:22:51 +00:00
Jan Malakhovski 845fa56921 stdenv: cleanup things a little bit 2018-04-26 20:15:51 +00:00
Jan Malakhovski d834ba6654 stdenv: introduce and use config.doCheckByDefault option 2018-04-26 20:15:47 +00:00
Jan Malakhovski 50af975d85 stdenv: implement checkTarget and installCheckTarget autodetection 2018-04-25 19:53:25 +00:00
Jan Malakhovski e9e06888ed stdenv: generic/setup.sh: cleanup installPhase 2018-04-25 19:53:25 +00:00
Jan Malakhovski ad98c36f1b stdenv: generic/setup.sh: simplify buildPhase Makefile check 2018-04-25 19:53:24 +00:00