Netatalk 4.0 - Future-proofing Apple File Sharing

[rdmark]

I pushed a few updates to the verbiage this morning. At the time I had a nice long response typed up, but it got lost due to some browser local storage mishap. Gah!

Good news: In a few months' time, it will soon no longer be true that you need to build from source to get AppleTalk support. Debian, Ubuntu, Fedora, Arch, and Slack have recently packaged netatalk 4.0 which will trickle out to stable distro releases throughout 2025.

Generally speaking, the point with Docker is that it isolates software from the host OS. Traditionally, you install software on a *NIX system by putting hundreds or thousands of files all over the file system, while depending on dozens of libraries of specific versions being available in specific locations (a.k.a. dependency hell). A Docker container on the other hand is an isolated and self-contained file system with access only to the host system's kernel. You bypass dependency hell altogether. This makes it superbly trivial to spin up, update, and tear down software. It's amazing once you get the hang of it.

The netatalk 4.0 Docker container can in fact do AppleTalk as long as the host OS is Linux that's not Red Hat or a Red Hat derivative. This is because the AppleTalk DDP transport layer is situated in the Linux kernel, and Red Hat has purged AppleTalk for security reasons. They don't like old, obscure network protocols.

I don't know if this answers your question though. I see Docker as a somewhat easier way to get started with Netatalk + AppleTalk compared to building from source, for someone who is a beginner with both.

 

[KennyPowers]

That's a great, concise explanation of Docker and the types of problems it solves I think when a lot of people first start learning about containers and Docker, the instinctive thing is to compare containers to virtual machines since they're conceptually similar in a lot of ways. An enlightening distinction then is that VMs virtualize hardware, while containers kind of just virtualize the OS.

 

[Mk.558]

Sounds kinda neat, like iOS in the earlier days with its sandboxy nature.

But shirley you don't get something for nothing. What kind of problems does it introduce? Every type of methodology has certain things it's good at, certain things it's not good at. Of course it's Linux, so the answers usually flip between between "it depends", "...it's complicated..." and "what distro you use".

 

[JDW]

 

[rdmark]

Ha!

The biggest drawback in my mind is the rigidity of the container once you build it. Unlike a normal Linux system, you can't just go in and, say, edit afp.conf and restart netatalk. When you restart the container its state will be reset and changes to afp.conf lost. This is by design, of course. It's how you can guarantee a stable and predictable container. If you need persistence of, say, afp.conf, you need to "bind" it (the file, or the directory that contains it) to a different storage unit (can be another container, or a path on your local file system). If you are familiar with *NIX file systems, you know that it's possible to mount file systems at arbitrary mount points on the directory tree. This is what's happening here.

In a sense, administering a container is more of an inderect activity compared to tinkering directly with a system. You describe to Docker how you want the container to be configured, and then Docker orchestrates it for you.

 

[rdmark]

Good news: netatalk 4 is now available as a Homebrew formula for macOS (and Linux, for you weirdos who use brew on Linux)

If you have brew installed already, just do:

brew update
brew install netatalk
sudo brew services start netatalk

If you're using an Intel Mac, this should work out of the box.

If you're on Apple Silicon, one additional manual step is required:

sudo cp /opt/homebrew/etc/pam.d/netatalk /usr/local/etc/pam.d

If you try it out, please let me know how it went! This my first attempt at creating a brew formula, so there may be other improvement areas.

 

[rdmark]

Netatalk 4.1.0 is available.

Release Netatalk 4.1.0 · Netatalk/netatalk

Netatalk 4.1.0 is available! The Netatalk team is proud to announce the first release in the Netatalk 4.1 release series. This version further improves interoperability on macOS hosts, makes the Ma...

github.com

The standout feature in this version is: When running on a macOS host, afpd now stores Extended Attributes metadata in a macOS native format, rather than as user.org.netatalk.Metadata. This enables seamless retrieval and storage of files with EA metadata onto a shared AFP volume. For instance, if you decompress a StuffIt archive with the Unarchiver on macOS onto a shared AFP volume, any Classic Mac OS file will get extracted with its resource fork data intact.

Also, if you're an OpenWrt fan, you would be pleased to know that it's easier than ever to build netatalk and run it with full AppleTalk support... on supported router hardware. Both netatalk and the OpenWrt kernel has been recently patched.

 

[NJRoadfan]

To clarify, Netatalk 4.1.0 can store/retrieve the resource fork of files natively on macOS hosts without the need of "._" AppleDouble files. Additionally, Netatalk will now natively read and sync any FinderInfo data (File Type/Creator data, labels, etc.) it finds on the host file system.

 

[rdmark]

A quick tally on the Netatalk v4 adoption in downstream distribution channels:
The OSes and 3rd party repos that now distribute a version of Netatalk 4 are...

1. Arch Linux
2. Debian GNU/Linux
3. Devuan GNU+Linux
4. Fedora
5. Homebrew
6. Kali Linux
7. Mageia Linux
8. NetBSD
9. OpenBSD
10. OpenWrt
11. PureOS
12. Raspberry Pi OS
13. Slackware
14. Ubuntu

In the works, that I know of, are...

1. NixOS (thanks @eric !)
2. MacPorts

Anyways, I'm very pleased with the adoption so far, only 4 months after release. The notable laggards are Alpine Linux and Gentoo, which still distribute the older 3.1, as well as FreeBSD which upgraded their port to 3.2 a few months ago.

 

[rdmark]

This is more of a niche project update than usual, but I'm proud to share that as of today I've put in place structures that allow anyone to translate the Netatalk manual to other languages, and have them published on the netatalk.io website in the future. (Or be packaged by distros that package localized docs, and then read locally.)

To summarize the technical details: Before, the manual was kept in XML format, and then transformed with DocBook XSL to roff man pages or html pages. DocBook is a fine and capable technology, but it's very old school, rigid, and slow. So I worked a few late night to convert the whole manual, all readmes, and the entire website to modern Markdown (using pandoc to automate the process) and then polished it until it shone.

A preview version of the html manual is online at https://netatalk.io/manual/en/

Additionally, since the early 2010s there's been a Japanese localization of the Netatalk manual that's been effectively operated as a separate project by two diligent community members. They abandoned the project a few years ago, after which I pulled it into Netatalk proper. However, their approach was to take a copy of the 10k+ lines of XML code and hard code the translations, so to speak. Needless to say, keeping the translation up to date has been extremely cumbersome.

So as part of the Markdown transition, I introduced po4a, a gettext based tool, to break translations out of the source code. Now it's as easy as editing three files, running po4a, and then go crazy with translation in the resulting po files.

If someone feels up for translating just under 1200 strings of technical documentation, here's a guide for getting started.

 

[JDW]

I've found ChatGPT more adept at translation than Google translate, which has come a long way over the years. Back when Google Translate just started out, it's Engish<->Japanese translations were laughable. These days, nobody is laughing, although at times native speakers can recognize a machine translation at work. But with ChatGPT, it's often hard to tell if it was a human translation. That's because you can give ChatGPT much needed context. Tell it what you are trying to achieve with your translation, and tell it to consider similar translations for that particular application, and assuming it has access to those other translations you specify, it can end up doing a pretty darned good job.

You can also feed ChatGPT text files. So for example, if you have a text file of strings which are mostly translated but some not, you can tell it to consider the E<->J already translated in the text file and then finish translating the rest.

AI only gets better from here, folks. It's worth considering for mundane tasks like this.

 

[rdmark]

I'm using Claude Sonnet 3.5 heavily via the Sourcegraph Cody plugin for my IDE as a coding assistant, which is really handy for drafting boilerplate code, changelogs, release notes, and so on. AI is also invaluable for breaking down and explaining convoluted logic in the Netatalk C code (for an amateur like me.)

FWIW, I have found Google Translate preferable over LLMs as a translation assistant for Netatalk documentation specifically. The docs are already dry and technical in the original language, interspersed with code examples, option names and terminal commands, not to mention the XML or Markdown formatting syntax. Google Translate reproduces all of that 100% faithfully. I can touch up the Japanese translation later when needed to make the sentence structures flow more naturally and with better word choice. ChatGPT does indeed produce better prose, so to speak, but it often gets fanciful with technical details and nuances. Fact checking 1000s of lines is more labor intensive than touching up awkward grammar, in my experience.

BTW it always breaks me up when the LLM starts inventing bugfixes and names of contributors who don't exist when drafting release notes.

 

[theirongiant]

The online manual indicates that AppleTalk would be available if the option -Dwith-appletalk=true is passed at build time.

Ummm, how does one even do that with Homebrew? Or is AppleTalk enabled by default in the Homebrew build? Also, the "pap" command is missing. I'm already running a real AppleTalk Internet Router from a Macintosh LC III, and I'd like to add my laser printer to the existing zone for #Marchintosh. I'm using a 2018 Intel Mac mini.

 

[NJRoadfan]

macOS hosts don't support AppleTalk/DDP.

 

[theirongiant]

macOS hosts don't support AppleTalk/DDP.

 

[theirongiant]

"...but if you want that, I'd recommend building on the _________ platform."

this is what I was hoping you would say. Any suggestions?

 

[NJRoadfan]

Only Linux and NetBSD have actively maintained support for AppleTalk /DDP.

 

[rdmark]

This page goes into some detail about the current state of AppleTalk support: https://netatalk.io/appletalk

 

[scj312]

I've been using Netatalk 4.0.x Docker container successfully for quite a while and ran into my first real issue with it today. For some reason, afpd got into a crash loop:

Interestingly, an existing AFP connection I had up to it from another machine never disconnected. Did Netatalk accidentally spawn multiple instances? I've since updated to 4.1.2 and the container came up again just fine. I didn't notice any item in the change logs that relates to whatever I ran into, but hopefully this log is helpful.

 

[rdmark]

I've been using Netatalk 4.0.x Docker container successfully for quite a while and ran into my first real issue with it today. For some reason, afpd got into a crash loop:

View attachment 20494

Interestingly, an existing AFP connection I had up to it from another machine never disconnected. Did Netatalk accidentally spawn multiple instances? I've since updated to 4.1.2 and the container came up again just fine. I didn't notice any item in the change logs that relates to whatever I ran into, but hopefully this log is helpful.

Interesting. The error here is that netatalk wasn't able to detect a usable network interface for DSI (AFP over TCP). Was networking functioning at all in the container in that state? For instance, exec into a shell in the container and try pinging other server etc. This might be something worth trying the next time you experience this.