libtool's libtool.m4 script assumes that `file` is available, and can
be found at `/usr/bin/file` (this path is hardwired). Furthermore,
the script with this assumption is vendored into the ./configure
scripts of an enormous number of packages. Without this commit, you
will frequently see errors like this during the configurePhase with
the sandbox enabled:
./configure: line 9595: /usr/bin/file: command not found
Due mostly to luck, this error does not affect native compiles on
nixpkgs' two most popular platforms, x86_64-linux and aarch64-linux.
However it will cause incorrect linker flag detection and a failure to
generate shared libraries for sandboxed cross-builds to a x86_64-linux
host as well as any sandboxed build (cross or native) for the following
hosts: x86_64-freebsd, *-hpux, *-irix, mips64*-linux, powerpc*-linux,
s390x-linux, s390x-tpf, sparc-linux, and *-solaris.
This commit fixes the problem by adding an extra line to fixLibtool()
in pkgs/stdenv/generic/setup.sh. This extra line will scan the
unpacked source code for executable files named "configure" which
contain the following text:
'GNU Libtool is free software; you can redistribute it and/or modify'
This text is taken to be an indicator of a vendored libtool.m4. When
it is found, the configure script containing it is subjected to `sed
-i s_/usr/bin/file_file_` which replaces all occurrences of
`/usr/bin/file` with `file`.
Additionally, the `file` package is now considered to be part of
`stdenv`. It has been added to `common-path.nix` so that the `file`
binary will be found in the `$PATH` of every build, except for the
bootstrap-tools and the first few stages of stdenv boostrapping.
Verified no regressions under:
nix-build --arg pkgs 'import ./. {}' ./lib/tests/release.nix
This commit allows the following commands to complete, which should
enable Hydra to produce bootstrap-files for mips64el:
nix-build \
--option sandbox true \
--option sandbox-fallback false \
pkgs/top-level/release-cross.nix \
-A bootstrapTools.mips64el-linux-gnuabi64.build
nix-build \
--option sandbox true \
--option sandbox-fallback false \
. \
-A pkgsCross.mips64el-linux-gnuabi64.nix_2_4
Instead of requiring the platforms be equal, use `isCompatible` to
determine if we can execute tests. The upside of this is that we now
can execute tests for natively cross compiled package sets like
pkgsStatic, pkgsLLVM and pkgsCross.musl64 etc.
the motivation for this is to simplify stdenv and ease the job of
reviewers due to them needing to tell contributors about the defacto
rule that configureFlags should be a list of strings
This warning logs when a package has no maintainers. It will stay silent
if `meta.maintainers` is not set at all, only complaining when it is an
empty list. In the future a separate warning could be added to allow for
that stricter behavior. Or this warning could be changed.
This will allow for adding more validity types in the future, such as a
warning type. (which is in the next commit in this series)
This is NOT a breaking change because validity.valid is never exposed
outside of `stdenv.mkDerivation`.
This isn't really desirable in general, but given that Nix itself
currently relies on this behaviour and that we don't want to break
backwards compatibility we should support it for now, maybe
deprecating it in the future.
This change is prompted by the following, admittedly cursed tarball:
```
> curl https://registry.npmjs.org/char-regex/-/char-regex-1.0.2.tgz 2>/dev/null \
| tar -ztv
drw-rw-rw- 0/0 0 2020-02-18 10:50 package
-rw-rw-rw- 0/0 297 2020-02-18 10:50 package/index.d.ts
-rw-rw-rw- 0/0 1920 2020-02-18 10:50 package/index.js
-rw-rw-rw- 0/0 1092 2020-01-31 11:31 package/LICENSE
-rw-rw-rw- 0/0 937 2020-02-18 10:51 package/package.json
-rw-rw-rw- 0/0 713 2020-02-18 10:50 package/README.md
```
The minimal reproducer for the issue is the following derivation trying
to work around the uid 0 issue with `dontMakeSourcesWritable = true`:
```nix
{ stdenv, fetchurl }:
stdenv.mkDerivation {
name = "test";
src = fetchurl {
sha1 = "d744358226217f981ed58f479b1d6bcc29545dcf";
url = "https://registry.npmjs.org/char-regex/-/char-regex-1.0.2.tgz";
};
dontMakeSourcesWritable = true;
installPhase = ''
cp -R . $out
'';
}
```
This currently fails in the following way:
```
these derivations will be built:
/nix/store/pc3jbydl0xcc8nrndf5xkf7hdhpgpb41-test.drv
building '/nix/store/pc3jbydl0xcc8nrndf5xkf7hdhpgpb41-test.drv'...
unpacking sources
unpacking source archive /nix/store/v9p98kqplf4kflmy91p0687xlvr6klb1-char-regex-1.0.2.tgz
source root is package
find: 'package/index.d.ts': Permission denied
find: 'package/index.js': Permission denied
find: 'package/LICENSE': Permission denied
find: 'package/package.json': Permission denied
find: 'package/README.md': Permission denied
/nix/store/6c47azxacncswc1pllzj28zfzqw40d7c-stdenv-linux/setup: line 1311: cd: package: Permission denied
builder for '/nix/store/pc3jbydl0xcc8nrndf5xkf7hdhpgpb41-test.drv' failed with exit code 1
error: build of '/nix/store/pc3jbydl0xcc8nrndf5xkf7hdhpgpb41-test.drv' failed
```
As you can see, the issue is that `$sourceRoot` isn't executable,
prohibiting the call to `cd`. This can be fixed by running
`chmod +x "${sourceRoot}"` before `cd` regardless of
`dontMakeSourcesWritable` in `unpackPhase` since if `chmod` fails, `cd`
would fail as well and we are out of options.
Verified that the workaround works locally.
Another thing to investigate is investigating if we should use
`--no-same-owner` for `tar` and if it helps in this case as well.
See also <https://github.com/Profpatsch/yarn2nix/issues/56>.
Flake users that use a command like `nix build nixpkgs#hello` on a
broken/insecure package will not be able to use an environment variable
to override that behavior, unless they pass `--impure` to the command.
Co-authored-by: pkharvey <kayharvey@protonmail.com>
Add `shellDryRun` to the generic stdenv and substitute it for uses of
`${stdenv.shell} -n`. The point of this layer of abstraction is to add
the flag `-O extglob`, which resolves#126344 in a more direct way.
Packages that use libtool run it as a wrapper around the linker.
Before calling the linker, libtool will determine what libraries would
be linked, and check if there's a corresponding libtool
archive (libfoo.la) file in the same directory . This file
contains extra information about the library. This is especially
important for static linking, because static archives don't contain
dependency information, so we need libtool to use the .la files to
figure out which libraries actually need to be linked against.
But in Nixpkgs, this has never worked. libtool isn't able to find any
libraries, because only the compiler wrapper knows how to find them,
and the compiler wrapper is opaque to libtool. This is why
pkgsStatic.util-linuxMinimal doesn't build prior to this patch — it
depends on libpam, which depends on libaudit, and if libtool can't
find the .la file, nothing will tell the linker to also link against
libaudit when linking libpam. (It was previously possible to build a
static util-linux, because linux-pam only recently had the audit
dependency added.)
There are a couple of ways we could fix this, so that libtool knows
where to look for .la files.
* Set LD_LIBRARY_PATH/DYLD_LIBRARY_PATH/whatever, which libtool will
examine. This would have major side effects though, because the
dynamic linker looks at it too.
* Inject libtool scripts with the appropriate information. That's
what I've done here. It was the obvious choice because we're
already finding and modifying the libtool scripts, to remove paths
outside the Nix store that libtool might check in unsandboxed
builds. Instead of emptying out the system paths, we can
repopulate it with our own library paths.
(We can't use a wrapper like we do for other tools in Nixpkgs, because
libtool scripts are often distributed in source tarballs, so we can't
just add a wrapped version of libtool as a dependency. That's why
there's already the fixLibtool function in stdenv.)
With this change, libtool is able to discover .la files, and
pkgsStatic.util-linuxMinimal can build again, linking correctly
against libpam and libaudit.
