Using a simple algorithm, convert the references to a path in to a
sorted list of dependent paths based on how often they're referenced
and how deep in the tree they live. Equally-"popular" paths are then
sorted by name.
The existing writeReferencesToFile prints the paths in a simple
ascii-based sorting of the paths.
Sorting the paths by graph improves the chances that the difference
between two builds appear near the end of the list, instead of near
the beginning. This makes a difference for Nix builds which export a
closure for another program to consume, if that program implements its
own level of binary diffing.
For an example, Docker Images. If each store path is a separate layer
then Docker Images can be very efficiently transfered between systems,
and we get very good cache reuse between images built with the same
version of Nixpkgs. However, since Docker only reliably supports a
small number of layers (42) it is important to pick the individual
layers carefully. By storing very popular store paths in the first 40
layers, we improve the chances that the next Docker image will share
many of those layers.*
Given the dependency tree:
A - B - C - D -\
\ \ \ \
\ \ \ \
\ \ - E ---- F
\- G
Nodes which have multiple references are duplicated:
A - B - C - D - F
\ \ \
\ \ \- E - F
\ \
\ \- E - F
\
\- G
Each leaf node is now replaced by a counter defaulted to 1:
A - B - C - D - (F:1)
\ \ \
\ \ \- E - (F:1)
\ \
\ \- E - (F:1)
\
\- (G:1)
Then each leaf counter is merged with its parent node, replacing the
parent node with a counter of 1, and each existing counter being
incremented by 1. That is to say `- D - (F:1)` becomes `- (D:1, F:2)`:
A - B - C - (D:1, F:2)
\ \ \
\ \ \- (E:1, F:2)
\ \
\ \- (E:1, F:2)
\
\- (G:1)
Then each leaf counter is merged with its parent node again, merging
any counters, then incrementing each:
A - B - (C:1, D:2, E:2, F:5)
\ \
\ \- (E:1, F:2)
\
\- (G:1)
And again:
A - (B:1, C:2, D:3, E:4, F:8)
\
\- (G:1)
And again:
(A:1, B:2, C:3, D:4, E:5, F:9, G:2)
and then paths have the following "popularity":
A 1
B 2
C 3
D 4
E 5
F 9
G 2
and the popularity contest would result in the paths being printed as:
F
E
D
C
B
G
A
* Note: People who have used a Dockerfile before assume Docker's
Layers are inherently ordered. However, this is not true -- Docker
layers are content-addressable and are not explicitly layered until
they are composed in to an Image.
/cc @Ericson2314
PR was https://github.com/NixOS/nixpkgs/pull/46857
This line broke MacOS cross compilation. paxctl cannot be built on
macOS. Maybe it can be fixed, but no reason to break things
unnecessarily.
Regardless, you definitely need to be more careful about backporting.
I think it’s fine to move fast and break things on master but
with release-18.09 we should be more careful. Something like more
automated testing for cross compilation would also be
helpful (hopefully even making it block).
(cherry picked from commit f9c4075873cb56464126f993d22a1a72f7cfac45)
/cc @Ericson2314
This line broke MacOS cross compilation. paxctl cannot be built on
macOS. Maybe it can be fixed, but no reason to break things
unnecessarily.
Regardless, you definitely need to be more careful about backporting.
I think it’s fine to move fast and break things on master but
with release-18.09 we should be more careful. Something like more
automated testing for cross compilation would also be
helpful (hopefully even making it block).
Thanks to @symphorien for this work, which apart from the update itself
includes a few more fixes and cleanups.
I've tested building and running the upgraded Paperwork and while I
haven't done extensive testing on every little feature it seems to work
so far.
The changes also include an addition to fetchFromGitLab, which allows to
specify a group.
Merges: #46487
Similar to some other builds, sdl-config gets stuck and times out.
checking for sdl-config... /nix/store/q2hm1ka0qxs2gv5bmrxj7j80lygk4b5z-SDL-1.2.15-dev/bin/sdl-config
building of '/nix/store/m2d7v0n5hd2498vfxp18i37p7r1lf76p-freedroidrpg-0.16.1' timed out after 7200 seconds of silence
/cc ZHF #45961
Doesn't build on darwin and probably other platforms.
GsWidget.H:26:3: error: 'Atom' does not name a type; did you mean 'tm'?
Atom atoms[5];
^~~~
tm
GsWidget.cxx: In member function 'void GsWidget::setProps()':
GsWidget.cxx:47:2: error: 'atoms' was not declared in this scope
atoms[0] = XInternAtom(fl_display,"GHOSTVIEW" , false);
^~~~~
/cc ZHF #45961
Undefined symbols for architecture x86_64:
"_MPCreateSemaphore", referenced from:
FX::FXSemaphore::FXSemaphore(int) in FXThread.o
FX::FXSemaphore::FXSemaphore(int) in FXThread.o
"_MPDeleteSemaphore", referenced from:
FX::FXSemaphore::~FXSemaphore() in FXThread.o
FX::FXSemaphore::~FXSemaphore() in FXThread.o
"_MPSignalSemaphore", referenced from:
FX::FXSemaphore::post() in FXThread.o
"_MPWaitOnSemaphore", referenced from:
FX::FXSemaphore::wait() in FXThread.o
FX::FXSemaphore::trywait() in FXThread.o
ld: symbol(s) not found for architecture x86_64
/cc ZHF #45961
* needs at least qtgraphicaleffects not mentioned in docs
* doesn't want to use our libqmatrixclient, so let it use bundled
* took a few runs (or perhaps just patience with one) to populate,
when my profile icon appeared it seemed ready-- then click it
to see list of rooms and such.
* podofo: fix library linkage on Darwin
Because the library is not yet installed when the tools are build, it does not contain its final store path as its install name. Linking the tools picks up this incorrect install name and needs to be fixed after installing.
The patchPhase wasn't being applied at all.
This patch re-enables that and also re-thinks the setting that we want
to have. Turning off the auto-update is more accurate and doesn't lose
information like it did before.
- build fails with qt511 and qt59
(it used to build with qt510 which is not available anymore)
- no nixpkgs maintainer
- upstream repo looks inactive, last commit a year ago
- New dependency on 'getconf' binary for
3aa619e9ef/src/vm_memory_monitor.erl (L448)
- New dependency on 'socat' for systemd notifications
4a3ee3a336/src/rabbit.erl (L361)
- elixir_1_6 for a new 'rabbitmqctl' tool
- Replace patching with providing custom PATH, as we already have some
other things here
- Renamed package in all-packages.nix from a legacy spelling
This reverts commit f777d2b719.
cc #34409
This breaks evaluation of the tested job:
attribute 'diskInterface' missing, at /nix/store/5k9kk52bv6zsvsyyvpxhm8xmwyn2yjvx-source/pkgs/build-support/vm/default.nix:316:24
Only apply w/musl since while it's wrong everywhere it apparently
hasn't broken things entirely w/glibc so keep things as they were.
Patch regenerated from original so that it applies
which isn't saying much since it's simple :).
Source:
https://patchwork.ozlabs.org/patch/154298/
IRC chat on #musl with Rich and others endorses this,
at least at the conceptual level of no shared library
should be using initial-exec TLS.
Fixes various uses of libgomp that previously crashed (before 1.1.20)
or encounter errors (post-1.1.20), such as pythonPackages.cython .
This includes the initialy commit was done by @Mic92 plus a few fixes
from my side. So essentially this avoids patching statically linked
executables and also speeds up searching for ELF files altogether.
I've tested this by comparing the outputs of all the derivations which
make use of this hook using the following Nix expression:
let
getPackagesForRev = rev: with import (builtins.fetchGit {
url = ./.;
inherit rev;
}) { config.allowUnfree = true; }; [
cups-kyodialog3 elasticsearch franz gurobi javacard-devkit
masterpdfeditor maxx oracle-instantclient powershell reaper
teamviewer unixODBCDrivers.msodbcsql17 virtlyst wavebox zoom-us
];
pkgs = import <nixpkgs> {};
baseRev = "ef764eb0d8314b81a012dae04642b4766199956d";
in pkgs.runCommand "diff-contents" {
chset = pkgs.lib.zipListsWith (old: new: pkgs.runCommand "diff" {
inherit old new;
nativeBuildInputs = [ pkgs.nukeReferences ];
} ''
mkdir -p "''${NIX_STORE#/}"
cp --no-preserve=all -r "$old" "''${NIX_STORE#/}"
cp --no-preserve=all -r "$new" "''${NIX_STORE#/}"
find "''${old#/}" "''${new#/}" \
\( -type f -exec nuke-refs {} + \) -o \( -type l -delete \)
mkdir "$out"
echo "$old" > "$out/old-path"
echo "$new" > "$out/new-path"
diff -Nur "''${old#/}" "''${new#/}" > "$out/diff" || :
'') (getPackagesForRev baseRev) (getPackagesForRev "");
} ''
err=0
for c in $chset; do
if [ -s "$c/diff" ]; then
echo "$(< "$c/old-path") -> $(< "$c/new-path")" \
"differs, report: $c/diff" >&2
err=1
fi
done
[ $err -eq 0 ] && touch "$out"
''
With these changes there is only one derivation which has altered
contents, which is "franz". However the reason why it has differing
contents is not directly because of the autoPatchelfHook changes, but
because the "env-vars" file from the builder is in
"$out/opt/franz/env-vars" (Cc: @gnidorah) and we now have different
contents for NIX_CFLAGS_COMPILE and other environment variables.
I also tested this against a random static binary and the hook no longer
tries to patch it.
Merges: #47222
The unfree variant of elasticsearch uses autoPatchelfHook and since we
removed the dependency on file for the hook itself in
58a97dfb49 we no longer have zlib
propagated.
So we need to explicitly state that dependency here.
Signed-off-by: aszlig <aszlig@nix.build>
Cc: @apeschar, @basvandijk
The "maxx" package recursively runs isExecutable on a bunch of files and
since the change to use "readelf" instead of "file" a lot of errors like
this one are printed during build:
readelf: Error: Not an ELF file - it has the wrong magic bytes at the
start
While the isExecutable was never meant to be used outside of the
autoPatchelfHook, it's still a good idea to silence the errors because
whenever readelf fails, it clearly indicates that the file in question
is not a valid ELF file.
Signed-off-by: aszlig <aszlig@nix.build>
If the ELF file is not an executable, we do not get a PT_INTERP section,
because after all, it's a *shared* library.
So instead of checking for PT_INTERP (to avoid statically linked
executables) for all ELF files, we add another check to see if it's an
executable and *only* skip it when it is and there's no PT_INTERP.
Signed-off-by: aszlig <aszlig@nix.build>
Fixes a critical issue with macOS
[NEWS](https://bitbucket.org/mituharu/emacs-mac/raw/master/NEWS-mac)
* emacs-26.1-mac-7.2 (2018-09-09)
** Fixed bugs
*** Buffer contents are not displayed on macOS 10.14.
This is mainly because now NSViews are backed by Core Animation Layer
(layer-backed) by default and non-deferred drawing into views no
longer works. Instead of switching to deferred drawing (i.e., draw
only inside -[NSView drawRect:]), we draw into our own backing bitmap
in a non-deferred way as before, and update the view contents with the
resulting image via -[NSView updateLayer]. This "application-side
double buffering" is also available on OS X 10.8 - macOS 10.13 if you
set the frame parameter `inhibit-double-buffering' to nil when
creating a frame. Just like on macOS 10.14, such a frame does not do
LCD smoothing.
*** Screenshot grabbed via Services is displayed in wrong size when we
have display mirroring between Retina and non-Retina displays.
*** Cursor movement just after frame resize sometimes leaves garbage.
*** Crash by the Fall_threads call from the GUI thread at the select
emulation when there are multiple Lisp threads.
*** Info title has ASCII underline unlike other window systems.
*** Vertical scroll bar is created as horizontal if frame font height
is short.
** Improvements
*** macOS 10.14 adds property :appearance to (mac-application-state).
The value may be "NSAppearanceNameAqua" or "NSAppearanceNameDarkAqua".
*** Add new color format "mac:COLOR-LIST-NAME:COLOR-NAME" and
"mac:COLOR-NAME" (shorthand for "mac:System:COLOR-NAME"). The actual
color may be different depending on the global appearance setting on
macOS 10.14. For example, "mac:textColor" is black on the Light Mode
but is white on the Dark Mode.
*** Default frame colors respect appearance setting on macOS 10.14.
Now the default frame foreground/background color is
"mac:textColor"/"mac:textBackgroundColor", respectively. Changes of
the system setting of the global appearance are dynamically reflected.
*** New function `mac-color-list-alist' to get the available
combinations of COLOR-LIST-NAMEs and COLOR-NAMEs. Note that this
value is dependent on user environment and OS version. Also, some
combinations may represent image patterns rather than colors. For the
former cases, `(color-values "mac:COLOR-LIST-NAME:COLOR-NAME")'
returns nil.
The compilers themselves can pull them from `bootPkgs`, where they
should always come from anyways. This enforces that, simplifies that
code, and allows use to avoid more `rec { ... }` too.
Semi-automatic update generated by https://github.com/ryantm/nixpkgs-update tools.
This update was made based on information from https://repology.org/metapackage/libtins/versions.
These checks were done:
- built on NixOS
- Warning: no binary found that responded to help or version flags. (This warning appears even if the package isn't expected to have binaries.)
- found 4.0 with grep in /nix/store/izz1bkj35g5bb6kvqa5kqgb376xhardg-libtins-4.0
- found 4.0 in filename of file in /nix/store/izz1bkj35g5bb6kvqa5kqgb376xhardg-libtins-4.0
- directory tree listing: https://gist.github.com/26990576c9472ca38b4d21613157856b
The `overrideScope` bound by `makeScope` (via special `callPackage`)
took an override in the form `super: self { … }`. But this is
dangerously close to the `self: super { … }` form used by *everything*
else, even other definitions of `overrideScope`! Since that
implementation did not even share any code either until I changed it
recently in 3cf43547f4, this inconsistency
is almost certainly an oversight and not intentional.
Unfortunately, just as the inconstency is hard to debug if one just
assumes the conventional order, any sudden fix would break existing
overrides in the same hard-to-debug way. So instead of changing the
definition a new `overrideScope'` with the conventional order is added,
and old `overrideScope` deprecated with a warning saying to use
`overrideScope'` instead. That will hopefully get people to stop using
`overrideScope`, freeing our hand to change or remove it in the future.
nfs-utils had a dependency on gcc through
etc/systemd/system-generators/*-server-generator. It was not stripped
correctly because it’s not in an expected path. This adds it to the
strip list.
Update to the latest release.
Highlights for c-lightning users
--------------------------------
- Less stuck payments: Liveness ping test before locking up funds with peers.
- Better routing: now considers size of channels.
- Fewer spurious closes: fee estimate improvements, and new feerates command
- Several annoying bugs fixed.
Highlights for the network
--------------------------
- Gossipd now less spammy with channel_update.
- option_data_loss_protect to protect peers against being out-of-date.
- Payment errors now refer to the correct channel.
Internal Improvements
---------------------
- Simplified client flow; after init message exchange by connectd, each is
isolated in its own daemon.
- JSON parameter handling vastly simplfied.
- Python testing framework now uses proper fixtures, and split into separate
files.
- Many other cleanups and clarifications.
- We keepachangelog.com!
Signed-off-by: William Casarin <jb55@jb55.com>
Now ARMv6 and ARMv7 builds avoid installing DTBs for boards where the
resulting kernel won't run just like the aarch64 builds already do.
Also deduplicates the logic a bit.
See https://hydra.nixos.org/build/80705583
Recent `boost` versions with `python` enabled have changed their naming
scheme for `boost_python` shared objects which causes issues with the
proper linking when building `pyftgl`.
Previously the library was named `boost_python3`, no it's named
`boost_python36` for current python (3.6.x).
The same issue applies for the `python2`.
Addresses #45960
Fixes the issue: https://github.com/NixOS/nixpkgs/issues/39149
Problem was that the Nvidia driver did not find the libxcb-glx at runtime.
(cherry picked from commit bda072cafcef4cf5ff99828852ddc8e06ce1fdbf)
The ZSH completion script of `tig` basically imports the bash completion
script and uses it as completion approach. Unfortunately the script
takes several assumptions about the directory structure using
`$funcsourcetrace`[1] that don't apply on NixOS.
The easiest workaround is t opatch the completion script and import the
bash completion script from `$out`.
[1] http://zsh.sourceforge.net/Doc/Release/Zsh-Modules.html#index-funcsourcetrace
2a6e4ae49a and
e51f736076 reverted a bit too much, and I
initially missed this when reviewing. The release notes already still
mention this change, too.
PHP tries to discover the mysql default socket path during configure
phase by probing the file system:
cf3b852109/ext/mysqli/config.m4 (L4)
This obviously fails to discover /run/mysqld/mysqld.sock, which is being
used (hardcoded) across all MySQL flavours.
This leads to PHP having no mysql socket path set for the mysql[i]
extensions, and `/tmp/mysql.sock` set for pdo_mysql,
meaning one currently has to manually configure and set it in php.ini.
Luckily, PHP supports setting that path via
`--with-mysql-sock=/run/mysqld/mysqld.sock` during configure phase,
so let's do this as soon as one of the three modules is enabled.
As per the following bug report, sometimes erts/start_scripts will fail
to compile because of a Makefile ordering issue. Apply the upstream
patches to fix this.
https://bugs.erlang.org/browse/ERL-241
This isn't necessary and causes build failures for certain versions.
CC libavfilter/libmpcodecs/vf_eq2.o
CC libavfilter/libmpcodecs/vf_fil.o
CC libavfilter/libmpcodecs/vf_fspp.o
libavfilter/libmpcodecs/vf_fspp.c:939:18: error: unknown token in expression
movq %mm2, 0*8+(%rsp)
/cc ZHF #45961
This way we don't need to disable flags etc by platform and can still
backport new versions to stable for linux even if there's a bug or
something in the darwin build.
