Commit graph

108 commits

Author SHA1 Message Date
stuebinm ff1a94e523 treewide: add meta.mainProgram to packages with a single binary
The nixpkgs-unstable channel's programs.sqlite was used to identify
packages producing exactly one binary, and these automatically added
to their package definitions wherever possible.
2024-03-19 03:14:51 +01:00
Lily Foster ae86a507ed npmHooks.npmInstallHook: ignore bundle deps when calculating files to install
This assumes that downstream users of `buildNpmPackage` would rather our
own built `node_modules` be copied to the output rather than only the
`bundleDependencies` specified in the `package.json` file.

Having the latter behavior seems unexpected and unintuitive, and would
not work as installing from an `npm pack` is intended to (since doing
that would not do a `rebuild` step on those dependencies and it would
skip reifying a full dependency tree).
2024-03-14 14:04:51 +01:00
adisbladis b6e4b86809 importNpmLock: init
This is an alternative to `fetchNpmDeps` that is notably different in that it uses metadata from `package.json` & `package-lock.json` instead of specifying a fixed-output hash.

Notable features:
- IFD free.
- Only fetches a node dependency once. No massive FODs.
- Support for URL, Git and path dependencies.
- Uses most of the existing `npmHooks`

`importNpmLock` can be used _only_ in the cases where we need to check in a `package-lock.json` in the tree.
Currently this means that we have 13 packages that would be candidates to use this function, though I expect most usage to be in private repositories.

This is upstreaming the builder portion of https://github.com/adisbladis/buildNodeModules into nixpkgs (different naming but the code is the same).
I will archive this repository and consider nixpkgs the new upstream once it's been merged.

For more explanations and rationale see https://discourse.nixos.org/t/buildnodemodules-the-dumbest-node-to-nix-packaging-tool-yet/35733

Example usage:
``` nix
stdenv.mkDerivation {
  pname = "my-nodejs-app";
  version = "0.1.0";

  src = ./.;

  nativeBuildInputs = [
    importNpmLock.hooks.npmConfigHook
    nodejs
    nodejs.passthru.python # for node-gyp
    npmHooks.npmBuildHook
    npmHooks.npmInstallHook
  ];

  npmDeps = buildNodeModules.fetchNodeModules {
    npmRoot = ./.;
  };
}
```
2024-03-05 12:23:28 +13:00
Lily Foster 8999c9d930 npmHooks.npmInstallHook: ignore rather than error for missing bin/man keys in package.json 2023-12-24 22:58:32 +01:00
Lily Foster b5ca84b450 npmHooks.npmInstallHook: only overwrite npm cache for npm pack rather than for entire hook 2023-12-05 21:53:59 +01:00
Lily Foster 63fabdebd0
Merge pull request #267912 from reinismu/fetch-npm-deps-fix-bug
prefetch-npm-deps: add support for npm alias schema in version spec
2023-12-05 14:39:04 -05:00
Janik b678419eac
Merge pull request #269078 from lilyinstarlight/fix/prefetch-npm-deps-more-bad-lockfiles 2023-12-03 06:53:59 +01:00
Reinis Muiznieks 02dd7c7bb3 prefetch-npm-deps: add support for npm alias schema in version spec 2023-11-29 13:14:59 +02:00
Lily Foster 25596bd419
Merge pull request #257337 from lorenzleutgeb/yarn-file
prefetch-yarn-deps: Fix handling of scoped packages
2023-11-26 16:49:33 -05:00
Lily Foster 09081aa859
fetchNpmDeps: add test case where empty default lockfile packages is needed 2023-11-23 20:52:00 -05:00
Lily Foster 77571a847f
prefetch-npm-deps: use default value when lockfile has no deps 2023-11-23 20:51:59 -05:00
Lily Foster daec4bf734
prefetch-npm-deps: instrument some logging 2023-11-23 20:51:58 -05:00
Lily Foster 81ed58b0fe
prefetch-npm-deps: make cargo happy 2023-11-23 20:51:22 -05:00
Lily Foster ba656ad84e
prefetch-npm-deps: bump deps 2023-11-23 20:51:14 -05:00
Nick Cao a68bc4feaf
Merge pull request #266296 from yayayayaka/element-1.11.48
element-{web,desktop}: 1.11.47 -> 1.11.50
2023-11-23 09:27:14 -05:00
Janik 8e2c9abca8
Merge pull request #243458 from lilyinstarlight/fix/prefetch-npm-deps-error-when-no-cached-deps 2023-11-23 13:55:03 +01:00
Sandro Jäckel f16843cb31
fetch-yarn-deps: fix missing cert when fetching packages
Found while updating element-web
2023-11-22 21:25:36 +01:00
Lily Foster 8e3009d95c
buildNpmPackage: add forceEmptyCache option 2023-11-22 15:18:49 -05:00
Lily Foster 05dc145e80
fetchNpmDeps: add forceEmptyCache option 2023-11-22 15:18:39 -05:00
Lily Foster ec51a56dfc
prefetch-npm-deps: detect and error out when generating an empty cache 2023-11-22 15:18:16 -05:00
Lorenz Leutgeb 29cf6a0422 prefetch-yarn-deps: Fix access to .resolved
... and simplify `prefetchYarnDeps`
2023-11-22 18:03:48 +01:00
Lorenz Leutgeb 3843224a55 prefetch-yarn-deps: Fix parsing of scoped packages 2023-11-22 18:03:06 +01:00
Lily Foster ab99231a36
prefetch-yarn-deps: add cacert to provide certificates during fetches
Ideally fetch-yarn-deps could do like some other fetchers and support
using SSL_CERT_FILE if it exists and also only verify integrity on FOD
hash unless using an empty/test hash.

But this should keep at least the same semantics as before the recent
Node.js change to stop using the built-in certificate store in favor of
the system one (which does not exist by default in the build sandbox).
2023-11-21 16:24:57 -05:00
midchildan 79459354cb buildNpmPackages: add missing deependencies 2023-11-17 07:58:58 +00:00
Felix Bühler 05a19f48a9
Merge pull request #267778 from lilyinstarlight/fix/yay-more-fixup-yarn-lock-stuff
prefetch-yarn-deps: re-add git hash for fixup
2023-11-16 21:06:38 +01:00
happysalada 7fb490674f buildNpmPackage: allow passing npmDeps 2023-11-16 11:57:48 +00:00
Lily Foster d02eb2d0e5
prefetch-yarn-deps: re-add git hash for fixup 2023-11-15 21:02:52 -05:00
Janik 582f1e7321
Merge pull request #254420 from lilyinstarlight/fix/npm-pack-pls-😭
npmHooks.npmInstallHook: avoid script output in npm pack command
2023-11-13 22:43:57 +01:00
Yt 5aaeafbe26
buildNpmPackage: make nodejs overridable (#265171)
* buildNpmPackage: allow nodejs to be passed as argument

* Update doc/languages-frameworks/javascript.section.md

Co-authored-by: Lily Foster <lily@lily.flowers>

---------

Co-authored-by: Lily Foster <lily@lily.flowers>
2023-11-09 16:19:24 +00:00
Weijia Wang 093f098d26
Merge pull request #260011 from lilyinstarlight/fix/prefetch-npm-deps-network-error-recovery
prefetch-npm-deps: read url bodies within the retry loop
2023-10-12 19:58:20 +02:00
Lily Foster 554e2412e0
prefetch-npm-deps: read url bodies within the retry loop 2023-10-10 19:40:43 -04:00
Lily Foster a6df5a7719
prefetch-npm-deps: bump deps 2023-10-09 11:37:16 -04:00
Doron Behar d090cd9af4
Merge pull request #258384 from doronbehar/buildNpmPackage-makeWrapperArgs
buildNpmPackage: support makeWrapperArgs
2023-10-08 12:38:33 +03:00
Lily Foster 57087b8346
Merge pull request #255984 from SuperSandro2000/patch-1
build-npm-package: give a hint when npm prune fails
2023-10-07 17:51:52 -04:00
Sandro 30992a8c51
buildNpmPackage: give a hint when npm prune fails
Co-authored-by: Lily Foster <lily@lily.flowers>
2023-10-07 21:53:56 +02:00
Doron Behar fc96f6d0fd buildNpmPackage: support makeWrapperArgs 2023-10-07 14:09:05 +03:00
Dan Buch 14f76a96e8
fetch-yarn-deps: warn on undefined expected hash
instead of rejecting, given that the expected hash may not be known/provided.
2023-09-27 00:17:40 +00:00
nicoo fab52fca51 treewide: sha512 → hash 2023-09-22 18:37:42 +02:00
Rick van Schijndel a31a3eca58
Merge pull request #251066 from lilyinstarlight/feature/prefetch-npm-deps-tokens
prefetch-npm-deps: add support for NIX_NPM_TOKENS env var
2023-09-12 17:31:26 +02:00
Lily Foster 7f76ac6e09
fetchNpmDeps: pass NIX_NPM_TOKENS as an impure env var 2023-09-11 16:50:17 -04:00
Lily Foster e271266179
prefetch-npm-deps: add support for NIX_NPM_TOKENS env var 2023-09-11 16:49:36 -04:00
Lily Foster 9b2ca17abc
npmHooks.npmInstallHook: avoid script output in npm pack command 2023-09-10 10:31:41 -04:00
sternenseemann bd374243c0 npmHooks: use adjacent packages, not buildPackages
Hooks are essentially implemented as special shell packages that run on
their respective host platform. When they are used, they appear as
nativeBuildInputs (as they need to be executed as part of the build of a
package using them) so are taken from buildPackages relative to the
derivation using them.

Since the override in buildNpmPackage nullifies splicing, we take
npmHooks from buildPackages manually.

Fixes pkgsCross.ghcjs.buildPackages.emscripten and thus
pkgsCross.ghcjs.haskellPackages.ghc.
2023-09-09 17:55:37 +02:00
Robert Schütz 96e5dce777 buildNpmPackage: use installManPage 2023-08-19 10:04:00 -07:00
Pierre Bourdon 28693fb1c3
buildNpmPackage: symlink manpages to the correct output directory 2023-08-18 12:41:05 +02:00
Lily Foster 785ed11d0a
prefetch-npm-deps: fix error typo and unnecessary name qualifier 2023-08-09 19:13:26 -04:00
MicroBlock c2b8bf7f4e
feat: implement redirect for fetch-yarn-deps
This would fix NixOS/nixpkgs#245415
2023-07-26 15:13:37 +08:00
Lily Foster 249efd2271
Merge pull request #223457 from lilyinstarlight/fix/fetch-yarn-deps-pulsar
fetchYarnDeps: account for more yarn.lock spec edge cases
2023-07-16 20:24:16 -04:00
Stanislav Asunkin 38ca04ed64 buildNpmPackage: remove redundant passthru.npmDeps 2023-07-15 10:29:55 +03:00
Lily Foster bee82b47b5
Merge pull request #240419 from lilyinstarlight/feature/prefetch-npm-deps-isahc
prefetch-npm-deps: use isahc instead of ureq
2023-07-08 14:18:07 +02:00