Netatalk 4.0 - Future-proofing Apple File Sharing

rdmark

Moderator
Staff member
Oct 3, 2021
164
230
43
@rdmark I went to try out the printing support with CUPS. The documentation for the Docker image says that the CUPS web interface should be running on port 631, but I don't seem to have anything running on that port. I'm using the host networking driver, so I shouldn't have to forward any ports from the container. Also, shelling into the running container and running lpstat suggests that CUPS isn't running:

Code:
/ # lpstat -r
scheduler is not running

What am I missing?
That's odd, because I spun up the latest Docker image now, and can access the CUPS administrative web app on port 631...

What do you get if you run `lpstat -p -d`?

I cannot properly test actually printing with papd right now though. All my equipment is packed up for an international move...
 

rdmark

Moderator
Staff member
Oct 3, 2021
164
230
43
Apologies, cups was running on my host which is where I got the admin web app on port 631 from.

Turns out, cupsd was indeed not running in the container. When I add `cupsd` to the entrypoint script, all seems to work again. I'll have the (very simple) fix in a next bugfix release.

It's odd though, I'm certain this worked before by just installing the cups package. Alpine's behavior might have changed.
 

KennyPowers

Active Tinkerer
Jun 27, 2022
281
319
63
@KennyPowers There's a bleeding edge Docker image available at the GitHub registry now (not on Docker Hub.)

https://github.com/Netatalk/netatalk/pkgs/container/netatalk

docker pull ghcr.io/netatalk/netatalk:sha-7d9fad7

If you have the means, please pull this image and see if it solves the problem for you.
I pulled that image, but looks like it's amd64 architecture and I'm on a Pi (arm). No worries though, I'll just wait for it to make its way to Docker Hub. I'm in no hurry to print...just wanted to make sure I wasn't doing something wrong (y)
 

rdmark

Moderator
Staff member
Oct 3, 2021
164
230
43
@KennyPowers Ah, of course. I haven't set up cross-compilation in the GitHub bleeding edge docker image job... It seems a bit wasteful to have an extra 10 minute plus job on every push and every PR. We already use some 15.000 minutes of GitHub Actions resources every month, and I don't know exactly what their threshold is for freeloader projects like Netatalk...

Anyhow, all that aside, I pushed a one-off test image to Docker Hub. Can you try this?

docker pull netatalk/netatalk:test
 

KennyPowers

Active Tinkerer
Jun 27, 2022
281
319
63
Anyhow, all that aside, I pushed a one-off test image to Docker Hub. Can you try this?

docker pull netatalk/netatalk:test
Looks like it worked. I'll either need to enable remote administration with cupsctl or set up an SSH tunnel to access the web administration page from another computer on the LAN (no browser or windowing environment on the PI), but it appears to be running:

Code:
pi3bplus:~ $ curl localhost:631
<!DOCTYPE HTML>
<html>
  <head>
    <link rel="stylesheet" href="/cups.css" type="text/css">
    <link rel="shortcut icon" href="/apple-touch-icon.png" type="image/png">
    <meta charset="utf-8">
    <meta http-equiv="Content-Type" content="text/html; charset=utf-8">
    <meta http-equiv="X-UA-Compatible" content="IE=9">
    <meta name="viewport" content="width=device-width">
    <title>Home - CUPS 2.4.9</title>
  </head>
  <body>
    <div class="cups-header">
      <ul>
        <li><a href="https://openprinting.github.io/cups/" target="_blank">OpenPrinting CUPS</a></li>
        <li><a class="active" href="/">Home</a></li>
        <li><a href="/admin">Administration</a></li>
        <li><a href="/classes/">Classes</a></li>
        <li><a href="/help/">Help</a></li>
        <li><a href="/jobs/">Jobs</a></li>
        <li><a href="/printers/">Printers</a></li>
      </ul>
    </div>
    <div class="cups-body">
      <div class="row">
        <h1>OpenPrinting CUPS 2.4.9</h1>
        <p>The standards-based, open source printing system developed by <a class="jumbolink" href="https://openprinting.github.io/" target="_blank">OpenPrinting</a> for Linux® and other Unix®-like operating systems. CUPS uses <a href="https://www.pwg.org/ipp/everywhere.html" target="_blank">IPP Everywhere™</a> to support printing to local and network printers.</p>
      </div>
      <div class="row">
        <div class="thirds">
          <h2>CUPS for Users</h2>
          <p><a href="help/overview.html">Overview of CUPS</a></p>
          <p><a href="help/options.html">Command-Line Printing and Options</a></p>
        </div>
        <div class="thirds">
          <h2>CUPS for Administrators</h2>
          <p><a href="help/admin.html">Adding Printers and Classes</a></p>
          <p><a href="help/policies.html">Managing Operation Policies</a></p>
          <p><a href="help/network.html">Using Network Printers</a></p>
          <p><a href="help/firewalls.html">Firewalls</a></p>
          <p><a href="help/man-cupsd.conf.html">cupsd.conf Reference</a></p>
        </div>
        <div class="thirds">
          <h2>CUPS for Developers</h2>
          <p><a href="help/cupspm.html">CUPS Programming Manual</a></p>
          <p><a href="help/api-filter.html">Filter and Backend Programming</a></p>
        </div>
      </div>
    </div>
    <div class="cups-footer">Copyright &copy; 2021-2023 OpenPrinting. All rights reserved.</div>
  </body>
</html>

Thanks!
 

rdmark

Moderator
Staff member
Oct 3, 2021
164
230
43
Thanks for testing! If you're lucky, CUPS will register your printer automatically and papd will pick it up without user intervention. But some manual configuration may be involved, too. Good luck!
 

Mk.558

New Tinkerer
Nov 11, 2023
54
8
8
Last week I spun up a VM instance of Linux Mint 22 to try Netatalk from source code only, no Webmin work.

The version was 4.0.3 and it appeared to compile and build fine, but starting it would leave a variety of errors. I've nuked it and will try again. In the meantime I have some other observations/comments/questions:
  • In the list of prerequistes and dependencies, is libgcrypt supposed to be libgcrypt20?
  • The requirements on the main page, building from source page (of course different distros different requirements etc) and the README.md are all different. There is a requirement for the UnicodeData.txt, but no suggestion where to get it. I found it by digging through the unicode-org github file repository. Not sure where it should go...if I put it in /home/, then if another user logs in, he's got no access. Not in /var/, not in /srv/, ...
  • The text in the compile page here -- https://netatalk.io/stable/htmldocs/compile is all stacked with escape characters that means it's hard to change something if you want to. It's a tough call because it makes it easier to read by a human, but ...
  • Appears that a $ sudo is missing on $ meson install -C build
  • I'm curious about the DocBook choice. I decided to skip on DocBook to see what I could use Netatalk for what I'm looking for without instaling a large list of other stuff. Turns out that was a bad call because without it, the man pages don't get installed. It looks like DocBook can be used to export XML formatted files into PDF, HTML, man page format, or something else. I guess I'm confused: normal man pages work fine for this implementation in my opinion.
  • The log feature in afp.conf doesn't seem to work. Using the webmin on a 4.0.1 installation, I used the webmin to try to figure out the syntax, and using /var/log/, a file named netatalk in the field and choosing /var/log/netatalk.txt -- none of them worked in the webmin. Of course it works fine to dump to /var/log/syslog, but with regular use of Netatalk and other stuff on the same system I think that would be a bad idea.
I have a difficulty with getting macipgw working. Of all the Linux software suites that are capable of driving me insane, nftables is probably up there and it is not straightforward to figure out. This page https://web.archive.org/web/20161003004424/http://bodhizazen.net/Tutorials/iptables has information on how iptables works, but iptables is not the same thing as nftables and the whole thing is a spaghetti mess in a bucket, meaning the iptables info found here https://biosrhythm.com/?p=2791 won't work because most people have moved away from iptables.

I scoured the web for an example NAT configuration text file of any kind so I could try and make sense of it, but only found the that @NJRoadfan put up on his A2SERVER fork. It looks like this: https://github.com/NJRoadfan/a2server/blob/currentdev/files/macipgw-start.sh.txt

Copying that file out into a text file, making it executable, starting atalkd, then netatalk, then running that script seemed to be OK? If I use MacTCP, the Fetch says it gets an illegal PORT command from the 10.4 FTP server and aborts the connection. Remote FTP servers don't work at all. OpenTransport also doesn't work at all. Here's the VM at work:

Screenshot1.png

On this VM, I can connect to the FTP server, but can't do any commands like uploading/downloading/listing files. I think there's something wrong with the NAT service or something. This is kind of above my skill level: I can see that it did set up the tun0 interface and assigns the MacIP addressing, but that's about it.
 
Last edited:

rdmark

Moderator
Staff member
Oct 3, 2021
164
230
43
@Mk.558 Thanks for sharing your notes, as always! First a caveat: I've never used Mint. Wikipedia suggests that it's based on Ubuntu, so I am going to assume that it behaves the same.

In the list of prerequistes and dependencies, is libgcrypt supposed to be libgcrypt20?
In Debian and Ubuntu package repos, libgcrypt is an alias (virtual package) for libgcrypt20. Either one should work. I don't know if it's bad practice to use the virtual package?

The requirements on the main page, building from source page (of course different distros different requirements etc) and the README.md are all different. There is a requirement for the UnicodeData.txt, but no suggestion where to get it. I found it by digging through the unicode-org github file repository. Not sure where it should go...if I put it in /home/, then if another user logs in, he's got no access. Not in /var/, not in /srv/, ...
The best docs on this I have right now is under the Dependency Changes section in the 4.0.0 release notes:


But as you point out, this information didn't make into the manual. I'll make improvements to the docs in the next release!

The master copy of UnicodeData.txt is at: https://www.unicode.org/Public/UNIDATA/UnicodeData.txt

The easiest way to use it, is putting a copy into into the root of the Netatalk source dir. The build system should pick it up automatically. It is only a compile time dependency, not a run time dependency, so only the user that builds the software needs access.

The text in the compile page here -- https://netatalk.io/stable/htmldocs/compile is all stacked with escape characters that means it's hard to change something if you want to. It's a tough call because it makes it easier to read by a human, but ...
You mean the line break backslashes, right? The reason we do it like that is mostly to make it easier for us to edit and manage the sources in git. One idea I have right now is to do a regex search and replace when converting the yaml to xml to strip out the escape characters, then apply white-space: pre-wrap; style to the <pre> element to force line wrapping... But I'm just thinking out loud now. Let me tinker with it and see if I can make it look better.

Appears that a $ sudo is missing on $ meson install -C build
This is because these steps double as the continuous integration script. The particular GitHub runner in this case has only a root user, and no sudo command installed.

I'm curious about the DocBook choice. I decided to skip on DocBook to see what I could use Netatalk for what I'm looking for without instaling a large list of other stuff. Turns out that was a bad call because without it, the man pages don't get installed. It looks like DocBook can be used to export XML formatted files into PDF, HTML, man page format, or something else. I guess I'm confused: normal man pages work fine for this implementation in my opinion.
The Netatalk man pages were always generated from DocBook XML, since at least 2004 and maybe earlier. However before v4.0 there were canned troff pages under revision control that the maintainers kept up to date manually. I did away with that in v4.0 as an effect of moving fully to Meson (and to reduce manual labor overhead for every release.) This also removes the risk of shipping outdated or broken troff pages (which had happened in the past...)

As I alluded to in an earlier thread, I'm looking into moving to another source format for documentation. The whole XML toolchain is extremely large and cumbersome.

The log feature in afp.conf doesn't seem to work. Using the webmin on a 4.0.1 installation, I used the webmin to try to figure out the syntax, and using /var/log/, a file named netatalk in the field and choosing /var/log/netatalk.txt -- none of them worked in the webmin. Of course it works fine to dump to /var/log/syslog, but with regular use of Netatalk and other stuff on the same system I think that would be a bad idea.
Can you please share what you have on logging in your afp.conf?
This is what I usually set:

[Global] log level = default:debug log file = /var/log/netatalk.log

I scoured the web for an example NAT configuration text file of any kind so I could try and make sense of it, but only found the that @NJRoadfan put up on his A2SERVER fork. It looks like this: https://github.com/NJRoadfan/a2server/blob/currentdev/files/macipgw-start.sh.txt

Copying that file out into a text file, making it executable, starting atalkd, then netatalk, then running that script seemed to be OK? If I use MacTCP, the Fetch says it gets an illegal PORT command from the 10.4 FTP server and aborts the connection. Remote FTP servers don't work at all. OpenTransport also doesn't work at all. Here's the VM at work:

Did you read through https://github.com/Netatalk/netatalk/wiki/MacIP-Gateway ?

I wonder if your Mint OS is using a too old Linux kernel that still does the DDP filtering crap?

FWIW, I could only demonstrate macipgw tunneling on a bleeding edge Debian Trixie (Testing) and not on stable Debian Bookworm. I think you need a very recent Linux kernel with all of the recent appletalk patches, i.e. at least v6.7 or later, v6.9 or later recommended.
 
Last edited:

NJRoadfan

New Tinkerer
Feb 6, 2022
26
6
3
One significant block of scripting in A2SERVER's setup is recompiling the AppleTalk kernel module if IPDDP is found. Debian has always compiled IPDDP support in, while in my experience, Ubuntu has not. Don't know if that holds for forked distros.

The newer kernels aren't bug free either. I'm running into a regression with the broadcast bug fix that is causing extra broadcast response packets with a source address of 0.0 (almost always NBP Reply packets) instead of the machine's proper address. I think that patch needs some more work.
 

Mk.558

New Tinkerer
Nov 11, 2023
54
8
8
I get excited waiting for your responses every time rdmark and can't wait to hammer things out.

Linux Mint is Debian yes. It uses Ubuntu repositories for software updates, if you type $ sudo apt-get update it sources from Ubuntu updates, right now Noble. Kernel version reported by uname -a is ...

Code:
Linux frost-VirtualBox 6.8.0-45-generic #45-Ubuntu SMP PREEMPT_DYNAMIC Fri Aug 30 12:02:04 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Anyways. libgcrypt doesn't even show up in the cache, and if you type $ sudo apt-get install libgcrypt and then spam the TAB key to try to list what it shows, it only shows libgcrypt20, libgcrypt20-dev, libgcrypt-doc, and libgcrypt-mingw-w64-dev. I don't know. This is one repository, there's of course others.

mint22-1.png

I included the config file for the build system. That's a big config file. I'm assuming that when it says CONFIG_ATALK=m that means that AppleTalk is a module (obviously loaded with Netatalk) and if it said =n that you'd have to recompile the kernel? There's a number of places that say if it's not compiled with AppleTalk support, you need to recompile it, but not how to check.

You mean the line break backslashes, right? The reason we do it like that is mostly to make it easier for us to edit and manage the sources in git. One idea I have right now is to do a regex search and replace when converting the yaml to xml to strip out the escape characters, then apply white-space: pre-wrap; style to the <pre> element to force line wrapping... But I'm just thinking out loud now. Let me tinker with it and see if I can make it look better.

Yeah it's because I wanted to add -Dwith-webmin=true, and I was forced to manually de-escape character all that stuff out because it doesn't paste into a Terminal window cleanly. The same thing happens with QEMU unforunately...

I agree that the XML situation is a bit messy. Sure it's nice to have the option to export to PDF, HTML, ... but realistically you don't really need to do that. It's just a text file.

Here's the log feature I was talk--Oh there it is! I put that in there yesterday and it didn't show up in /var/log/. Let's check what's inside it -- oh it's empty. Maybe wrong ownership? Don't have write permissions?

netatalklog401.png

I changed the name to netatalk.log, ran $ sudo reboot now, then restarted atatlkd and netatalk. No changes. Yeap, the saves to afp.conf were saved and the log file was not generated. Keep in mind this is Netatalk 4.0.1.

And now for macipgw. It is too new to have IPDDP, the first image above shows that it's not included. Is there anywhere I can get a NAT configuration script kind of like what @NJRoadfan has? I think his script is fine, but it doesn't work properly. I'm going to re-do a Netatalk installation onto a fresh VM instance after this. Also can we have macipgw create its own debug log that just prints to /var/log/macipgw ? It's buggy and troublesome enough that if you use a script like NJRoadfan's then it doesn't print to the terminal and show what's going on anyways.
 

NJRoadfan

New Tinkerer
Feb 6, 2022
26
6
3
If macipgw is already running, that script might not run as it should. Type sudo systemctl stop macipgw before trying to run the script. The script itself requires root privileges, so add an sudo in front of it.

It may also not be picking up your default network interface. Replace $atalkd_if with the actual name of your ethernet interface (leave the quotes), it usually defaults to enp0s3 in VirtualBox. Also run the command sudo nftables list ruleset. It should output the following after the script is run:

Code:
table ip filter {
        chain INPUT {
                type filter hook input priority filter; policy accept;
        }

        chain FORWARD {
                type filter hook forward priority filter; policy accept;
                iifname "enp0s3" oifname "tun0" ct state established,related counter packets 0 bytes 0 accept
        }

        chain OUTPUT {
                type filter hook output priority filter; policy accept;
        }
}
table ip nat {
        chain PREROUTING {
                type nat hook prerouting priority dstnat; policy accept;
        }

        chain INPUT {
                type nat hook input priority srcnat; policy accept;
        }

        chain OUTPUT {
                type nat hook output priority dstnat; policy accept;
        }

        chain POSTROUTING {
                type nat hook postrouting priority srcnat; policy accept;
                oifname "enp0s3" counter packets 0 bytes 0 masquerade
        }
}
 

rdmark

Moderator
Staff member
Oct 3, 2021
164
230
43
I get excited waiting for your responses every time rdmark and can't wait to hammer things out.
I strive to not disappoint!

Linux Mint is Debian yes. It uses Ubuntu repositories for software updates, if you type $ sudo apt-get update it sources from Ubuntu updates, right now Noble. Kernel version reported by uname -a is ...

Code:
Linux frost-VirtualBox 6.8.0-45-generic #45-Ubuntu SMP PREEMPT_DYNAMIC Fri Aug 30 12:02:04 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Anyways. libgcrypt doesn't even show up in the cache, and if you type $ sudo apt-get install libgcrypt and then spam the TAB key to try to list what it shows, it only shows libgcrypt20, libgcrypt20-dev, libgcrypt-doc, and libgcrypt-mingw-w64-dev. I don't know. This is one repository, there's of course others.
Alrightie then, let's change to the canonical libgcrypt20 for Debian/Ubuntu across the board. A fix will be in the next release.

View attachment 18487

I included the config file for the build system. That's a big config file. I'm assuming that when it says CONFIG_ATALK=m that means that AppleTalk is a module (obviously loaded with Netatalk) and if it said =n that you'd have to recompile the kernel? There's a number of places that say if it's not compiled with AppleTalk support, you need to recompile it, but not how to check.
The AppleTalk README has some suggestions how to check for appletalk support.

Yeah it's because I wanted to add -Dwith-webmin=true, and I was forced to manually de-escape character all that stuff out because it doesn't paste into a Terminal window cleanly. The same thing happens with QEMU unforunately...

I agree that the XML situation is a bit messy. Sure it's nice to have the option to export to PDF, HTML, ... but realistically you don't really need to do that. It's just a text file.
The html format is actually important for us. This is how we publish the manual to the netatalk.io website. One of the strengths of DocBook is that you can seamlessly transcode the same XML sources to troff and html.

Here's the log feature I was talk--Oh there it is! I put that in there yesterday and it didn't show up in /var/log/. Let's check what's inside it -- oh it's empty. Maybe wrong ownership? Don't have write permissions?
There's a fault in your afp.conf. In 'log level' you have to specify the logtype (before the colon) as well as loglevel (after the colon). I recommend using my example from earlier verbatim:

Code:
log level = default:debug

The "default" logtype means, log everything. The afp.conf man page has a list of available logtypes.
 

Mk.558

New Tinkerer
Nov 11, 2023
54
8
8
More updates. Newly compiled 4.0.4 under Mint 22 once again.

No issues with the netatalk file server feature. I compiled for Webmin and got it all set up first, then built & compiled Netatalk. Seems like the Webmin module is still in beta status though:

setting a log path doesn't work.pngScreenshot at 2024-11-05 19-40-20.png

First image is after doing something simple, like changing the log file location. Simply errors out after saving.
Second image is today. Can't log in to it.

Build string:
Code:
meson setup build -Dbuildtype=release -Dwith-appletalk=true -Dwith-dbus-sysconf-path=/usr/share/dbus-1/system.d -Dwith-init-hooks=false -Dwith-init-style=debian-sysv,systemd -Dwith-pkgconfdir-path=/etc/netatalk -Dwith-tests=true -Dwith-testsuite=true -Dwith-unicode-data-path="/home/flake/Downloads/netatalk-4.0.4/ -Dwith-webmin=true

Dependencies:

Code:
sudo apt-get install libavahi-client-dev libdb-dev libevent-dev libgcrypt20-dev meson xsltproc docbook-xsl

I'm getting nowhere with macipgw. Using a script file, it tells me invalid IP address. It works from the command line though. nftables is ranking high up on my list of software not to like very quickly. Using the command line, I get an error on almost every single entry, usually a bash syntax error for something. Even this example right off the man page doesn't work:

Code:
flake@flake-VirtualBox:~$ sudo add chain inet mytable myin { type filter hook input priority filter; }
bash: syntax error near unexpected token `}'
/CODE]

I will keep trying, but this isn't really very interesting. The computer is using me more than I'm using it. Suppose I need to make a text file for the whole ruleset and diagnose it line by line according to the errors i get.
 

Mk.558

New Tinkerer
Nov 11, 2023
54
8
8
Ah. huh. Why does the man page not even mention that? The missing nft command was on me, i caught that earlier, but the syntax errors...man...

Here's what I get for the A2SERVER script:

Screenshot at 2024-11-05 20-06-49.png

Note the macipgw failure to start.
 

NJRoadfan

New Tinkerer
Feb 6, 2022
26
6
3
chmod +x the script file and run it directly. Also could be weird newline characters on the file screwing things up. Cutting and pasting the text into a nano session usually avoids this.
 

Mk.558

New Tinkerer
Nov 11, 2023
54
8
8
... looks clean to me in nano. The script was +x'd about an hour before this. maybe I missed something

Screenshot at 2024-11-05 20-23-16.png

edit: i got NAT rules set up manually by typing $ sudo and then pasting line by line the whole NAT ruleset...let's see what happens
 

NJRoadfan

New Tinkerer
Feb 6, 2022
26
6
3
A quick search indicates that Linux Mint might not use nftables for firewall usage. If so, this stuff isn't going to work without additional setup. Try typing nft flush ruleset from the command prompt and see if an error is generated.