Episode 267: JSJ 266 NPM 5.0 with Rebecca Turner

JavaScript Jabber

• Episode: Episode 267: JSJ 266 NPM 5.0 with Rebecca Turner
• Website: JavaScript Jabber
• Feed URL: https://feeds.feedwrench.com/JavaScriptJabber.rss
• Duration: 00:41:40
• Published: 2017-06-20 12:00

On today’s episode of JavaScript Jabber, Charles Max Wood and panelist Joe Eames chat with Rebecca Turner, tech lead for NPM, a popular Javascript package manager with the worlds largest software registry. Learn about the newly released NPM 5 including a few of the updated features. Stay tuned!

[1:58] Was the release of node JS 8 tied to NPM5?

• Features in NPM5 have been in planning for 2 years now.
• Planned on getting it out earlier this year.
• Node 8 was coming out and got pushed out a month.
• Putting NPM5 into Node 8 became doable.
• Pushed really hard to get NPM5 into Node 8 so that users would get NPM5 and updates to NPM5.

[2:58] Why would it matter? NPM doesn’t care right?

• Right you can use NPM5 with any version of node.
• Most people don’t update NPM, but upgrade Node.
• So releasing them together allowed for when people updated Node they would get NPM 5.

[3:29] How does the upgrade process work if you’re using NVM or some node version manager?

• Depends. Different approaches for each
• NVM gets a fresh copy of Node with new globals. NVM5 and Node 8 are bundled.
• For some, If you manually upgrade NVM you’ll always have to manually. It will keep the one you manually upgraded to.

[4:16] Why NPM 5?

• It’s night and day faster.
• 3 to 5 times speed up is not uncommon.
• Most package managers are slow.
• NPM 5 is still growing. Will get even faster.

[5:18] How did you make it faster?

• The NPM’s cache is old. It’s very slow. Appalling slow.
• Rewrote cache
• Saw huge performance gains

[5:49] What is the function of the cache?

• Cache makes it so you don’t have to reinstall modules from the internet.
• It has registry information too.
• It will now obey http headers for timing out cache.

[6:50] Other things that made it faster?

• Had a log file for a long time. It was called shrinkwrap.
• NPM 5 makes it default.
• Renamed it to packagelog.json
• Exactly like shrinkwrap package file seen before
• In combo with cache, it makes it really fast.
• Stores information about what the tree should look like and it’s general structure.
• It doesn’t have to go back and learn versions of packages.

[7:50] Can you turn the default Packagelog.json off?

• Yes. Just:
• Set packagelog=false in the npmrc

[8:01] Why make it default? Why wasn’t it default before?

• It Didn’t have it before. Shrinkwrap was added as a separate project enfolded in NPM and wasn’t core to the design of NPM.
• Most people would now benefit from it. Not many scenarios where you wouldn’t want one.
• Teams not using the same tools causes headaches and issues.

[9:38] Where does not having a lock show up as a problem?

• It records the versions of the packages installed and where NPM put them so that when you clone a project down you will have exactly the same versions across machines.
• Collaborators have the exact same version.
• Protects from issues after people introduce changes and patch releases.
• NPM being faster is just a bonus.
• Store the sha512 of the package that was installed in the glock file so that we can verify it when you install. It’s Bit for bit what you had previously.

[11:12] Could you solve that by setting the package version as the same version as the .Json file?

• No. That will lock down the versions of the modules that you install personally, not the dependancies, or transitive dependancies.
• Package log allows you to look into the head of the installer. This is what the install looks like.

[12:16] Defaulting the log file speed things up? How?

• It doesn’t have to figure out dependences or the tree which makes it faster.
• Shrinkwrap command is still there, it renames it to shrinkwrap but shrinkwrap cannot be published.
• For application level things or big libraries, using shrinkwrap to lock down versions is popular.

[13:42] You’ve Adopted specifications in a ROC process. When did you guys do that?

• Did it in January
• Have been using them internally for years. Inviting people into the process.
• Specifications
• Written in the form of “Here is the problem and here are the solutions.”
• Spec folder in NPM docs, things being added to that as they specify how things work.
• Spec tests have been great.

[14:59] The update adds new tools. Will there be new things in registry as well?

• Yes.
• Information about a package from registry, it returns document that has info about every version and package json data and full readme for every version.
• It gets very large.
• New API to request smaller version of that document.
• Reduces bandwidth, lower download size, makes it substantially faster.
• Used to be hashed with sha1, With this update it will be hashed with sha512 as well as sha1 for older clients.

[16:20] Will you be stopping support for older versions?

• LTS version of NPM was a thing for a while. They stopped doing that.
• Two models, people either use whatever version came with Node or they update to the latest.
• The NPM team is really small. Hard to maintain old NPM branches.
• Supports current versions and that’s pretty much it.
• If there are big problems they will fix old versions. Patches , etc.

[17:36] Will there ever be problems with that?

• Older versions should continue to work. Shouldn’t break any of that.
• Can’t upgrade from 0.8.
• It does break with different Node version
• Does not support Node versions 0.10 or 0.12.

[18:47] How do you upgrade to NPM?

• sudo npm install -gmpm
• Yes, you may not need sudo. depend on what you’re on.

[19:07] How long has it been since version 4?

• Last October is when it came out.

[19:24] Do you already have plans for version 6?

• Yes!
• More releases than before coming up.
• Finally deprecating old features that are only used in a few packages out of the whole registry.
• Running tests on getting rid of things.

[20:50] Self healing cache. What is it and why do we want it?

• Users are sometimes showing up where installs are broken and tarbols are corrupted.
• This happens sometimes with complicated containerization setups makes it more likely. It’s unclear where the problem actually is.
• CaCache - content addressable cache. Take the hash of your package and use it to look up address to look it up in the cache.
• Compares the Tarbol using an address to look it up in the cache.
• Compares to see if it’s old. Trashes old and downloads updated one.
• Came out with the cache. Free side effect of the new cache.

[23:14] New information output as part of the update?

• NPM has always gave back you the tree from what you just installed.
• Now, trees can be larger and displaying that much information is not useful.
• User patch - gives you specifically what you asked for.
• Information it shows will be something like: “I installed 50 items, updated 7, deleted 2.”

[24:23] Did you personally put that together?

• Yes, threw it together and then got feedback from users and went with it.
• Often unplanned features will get made and will be thrown out to get feedback.
• Another new things ls output now shows you modules that were deduped. Shows logical tree and it’s relationships and what was deduped.

[25:27] You came up to node 4 syntax. Why not go to node 8?

• To allow people with just node 4 be able to use NPM.
• Many projects still run Node 4. Once a project has been deployed, people generally don’t touch it.

[26:20] Other new features? What about the File Specifier?

• File specifier is new. File paths can be in package json, usually put inside pointing to something inside your package.
• It will copy from there to your node modules.
• Just a node module symlink.
• Much faster. Verifiable that what’s in your node modules matches the source. If it’s pointing at the right place it’s correct. If not, then it’s not.
• Earlier, sometimes it was hard to tell.

[27:38] Anything else as part of the NPM 5 release? Who do you think will be most affected by it?

• For the most part, people notice three things:
• 1st. no giant tree at the end
• 2nd. Much faster
• 3rd. Package lock.

[28:14] If it’s locked, how do you update it?

• Run npm installer and then npm update
• Used to be scary, but works well now.
• Updates to latest semver, matches semver to package json to all node modules.
• Updates package lock at the same time
• Summary in Git shows what’s changed.

[28:59] Did Yarn come into play with your decisions with this release?

• The plans have been in play for a long time for this update.
• Yarn’s inclusion of similar features and the feedback was an indicator that some of the features were valuable.

[29:53] Other plans to incorporate features similar to yarn?

• Features are already pretty close.
• There are other alternative package managers out there.
• PMPM interesting because when it installs it doesn’t copy all the files. It creates hard links.

[30:28] Does PMPM and Yarn use NPM registry?

• Yes! Other than CNPM. The NPM client used in China.
• CNPM Registry mirror behind firewall. Have their own client to their registry. Their registry is a copy of ours.

[31:15] What about RNPM?

• I wouldn’t be surprised.

[31:45] “Won’t you come and say something controversial about your competitor?”

• We all want it to be collaborative.
• When we were writing our new cache, we also helped Yarn with their cache and sped things up tremendously.

---

Picks

Charles

Rush Limbaugh’s children’s books
Tinker Crate
Kiwi Crate
NPM
Episodes on My JS Story.

Joe

Gravity Falls
Board Games

Rebecca

NPX

Funstream

---

Links to keep up with NPM and Rebecca

Twitter @rebeccaorg
NPMjS on Twitter
blog.npmjs.com

Special Guest: Rebecca Turner.