Netatalk 4.0 - Future-proofing Apple File Sharing

[scj312]

Interesting. The error here is that netatalk wasn't able to detect a usable network interface for DSI (AFP over TCP). Was networking functioning at all in the container in that state? For instance, exec into a shell in the container and try pinging other server etc. This might be something worth trying the next time you experience this.

Hopefully it doesn't happen again, but I will definitely do some more poking around before restarting the container next time. Thanks!

 

[scj312]

Interesting. The error here is that netatalk wasn't able to detect a usable network interface for DSI (AFP over TCP). Was networking functioning at all in the container in that state? For instance, exec into a shell in the container and try pinging other server etc. This might be something worth trying the next time you experience this.

Well, it has already happened again. A few things I have been able to confirm:
1. The container still has network access (I can ping both inside and outside of the network via docker exec)
2. In this state, I cannot see Netatalk via AppleTalk
3. My modern Mac which has an AFP share already mounted is not interrupted. I can continue to browse and open files on the share. However, I can't see the listing of shares in the Finder to open a new one.
4. I see multiple instances of afpd when running ps aux in the container:

PID   USER     TIME  COMMAND
    1 root      0:00 {entrypoint.sh} /bin/bash /entrypoint.sh
   21 root      0:00 cupsd -C /etc/cups/cupsd.conf -s /etc/cups/cups-files.conf
   23 root      0:06 atalkd
   27 root      0:00 papd
   29 root      0:00 timelord -l
   31 root      0:00 a2boot
   32 root      0:12 netatalk -d
   34 root      0:00 /usr/local/sbin/cnid_metad -d -F /usr/local/etc/afp.conf
   37 pluto     0:07 /usr/local/sbin/afpd -d -F /usr/local/etc/afp.conf
   44 root      0:00 /usr/local/sbin/cnid_dbd -F /usr/local/etc/afp.conf -p /mnt/afpshare -t 10 -l 5 -u pluto
  190 nobody    0:00 /usr/local/sbin/afpd -d -F /usr/local/etc/afp.conf
  191 root      0:00 /usr/local/sbin/cnid_dbd -F /usr/local/etc/afp.conf -p /mnt/globaltalk/PurrTopia Web ƒ -t 7 -l 5 -u nobody
 7455 root      0:00 ping 1.1.1.1
 7468 root      0:00 ping 10.0.1.1
 7474 root      0:00 ping 10.0.1.14
 7517 root      0:00 /usr/local/sbin/afpd -d -F /usr/local/etc/afp.conf
 7518 root      0:00 ps aux

FWIW, the only change I have made to my Netatalk setup recently has been to add some additional shares, but I haven't made any other changes (well, other than upgrading the container the other day). Here is my config: https://gist.github.com/dremin/5deaffb336828b0e9283ca7aa5fe90e3

 

[rdmark]

Could you try ejecting the connected AFP mount from the modern Mac, and then try connecting again to netatalk in the container through the Connect to Server dialog: Press Cmd+K in Finder and connect to "afp://ip-address-of-docker-host"

It would also be interesting if you could run "nbplkup" inside the container, while an old Mac is also running and connected to the network. This would tell us to what extent AppleTalk routing is working.

Do you remember which netatalk version you were running before, which was stable?

What host OS are you running this container on?

 

[scj312]

Could you try ejecting the connected AFP mount from the modern Mac, and then try connecting again to netatalk in the container through the Connect to Server dialog: Press Cmd+K in Finder and connect to "afp://ip-address-of-docker-host"

It would also be interesting if you could run "nbplkup" inside the container, while an old Mac is also running and connected to the network. This would tell us to what extent AppleTalk routing is working.

Do you remember which netatalk version you were running before, which was stable?

What host OS are you running this container on?

Connecting via IP address did not work in the failed state.

nbplkup in the failed state can still see my jrouter instance running on another host, I forgot to go turn on a Mac:

For versions, 4.0.4 was stable until it wasn't--that's the version I was on when I originally posted about the issue. However, it had been very stable for me until recently. The docker host is running Ubuntu Server 20.04.3 and Docker 20.10.21.

 

[scj312]

Connecting via IP address did not work in the failed state.

nbplkup in the failed state can still see my jrouter instance running on another host, I forgot to go turn on a Mac:
For versions, 4.0.4 was stable until it wasn't--that's the version I was on when I originally posted about the issue. However, it had been very stable for me until recently. The docker host is running Ubuntu Server 20.04.3 and Docker 20.10.21.

Alright, it failed again and this time I remembered to turn on some more Macs! Here is the nbplkup result within the container. I also ran some pings to IPs both on and off the subnet. It looks like everything is fine from a network perspective, so the errors from Netatalk are a bit puzzling.

 

[rdmark]

@scj312 Agreed that this is highly puzzling. Have you tried to do what the logs earlier suggested: define an explicit "afp listen" or "afp interfaces" in afp.conf (override with MANUAL_CONFIG plus a bind mount with a custom afp.conf)

Beyond that, what I'd suggest is to enable debug logging for netatalk and see if that gives us a hint exactly what's going on internally.

 

[rdmark]

quick poll: of you netatalk users here, is anyone relying on the "include = /path/to/extra_afp.conf" directive in afp.conf?

For context, this is a directive that allows you to dynamically load and nest a secondary afp.conf file. However this relied on local hacks to the iniparser library. I am on the fence whether to discard this feature, or reimplement it.

 

[scj312]

@scj312 Agreed that this is highly puzzling. Have you tried to do what the logs earlier suggested: define an explicit "afp listen" or "afp interfaces" in afp.conf (override with MANUAL_CONFIG plus a bind mount with a custom afp.conf)

Beyond that, what I'd suggest is to enable debug logging for netatalk and see if that gives us a hint exactly what's going on internally.

I added `afp interfaces` to my `afp.conf` and haven't seen the issue since!

quick poll: of you netatalk users here, is anyone relying on the "include = /path/to/extra_afp.conf" directive in afp.conf?

I don't, as my config files are not large or complex enough to warrant this.

 

[scj312]

@rdmark I went to update my Docker container to 4.2.0, but noticed that version is not available on Docker Hub; the latest is 4.1.2. Thought you might like to know in case something broke

 

[rdmark]

@scj312 Yeah, I've been too busy with my day job to properly test 4.2.0 running in the Docker container. I don't want to inadvertently destroy anyone's deployment.

In 4.2.0, I completely rewrote the entry point script in POSIX compliant shell script to remove the bash dependency (for size and security.) Plus all the other potentially breaking changes that we made to 4.2.0.

Anyways, thanks for your interest and attention! I promise to get to it soon.

 

[rdmark]

@scj312 a 4.2.0 container image has been pushed to Docker Hub. Please check it out and let me know if it works out for you!

 

[scj312]

@scj312 a 4.2.0 container image has been pushed to Docker Hub. Please check it out and let me know if it works out for you!

Awesome, thanks! It's working great here.

 

[bribri]

I've got an issue maybe someone here can help with!

I'm trying to set up an AFP share in modern macOS that I connect to from Mac OS 7.5, and I want the contents of this share to be able to be added to a git repo. Since git doesn't handle extended attributes at all, I'm trying to set up the share to use AppleDouble metadata files for resource forks and such. However, if I specify "ea = ad" for the volume in afp.conf, it doesn't have any effect. No matter what I put there, it always uses extended attributes. Is there any way to force it to use AppleDouble files?

 

[rdmark]

@bribri What version of netatalk are you running? Would you be able to share the rest of your afp.conf?

You are using the correct option for forcing appledouble dot files so there must be an error elsewhere.

 

[joevt]

I've got an issue maybe someone here can help with!

I'm trying to set up an AFP share in modern macOS that I connect to from Mac OS 7.5, and I want the contents of this share to be able to be added to a git repo. Since git doesn't handle extended attributes at all, I'm trying to set up the share to use AppleDouble metadata files for resource forks and such. However, if I specify "ea = ad" for the volume in afp.conf, it doesn't have any effect. No matter what I put there, it always uses extended attributes. Is there any way to force it to use AppleDouble files?

I would probably write a script to handle the conversion between the git repository and the classic macOS share.
- Conversion between Mac Roman and UTF-8. Git doesn't like Mac Roman and CodeWarrior doesn't like UTF-8.
- Conversion between CR and LF. Git requires LF for proper diffs and merging. CodeWarrior can handle LF but SimpleText and MPW maybe not?
- Conversion between extended attributes and Apple double. CodeWarrior needs type/creator codes for default compilers. Also, they make opening files in classic macOS easier.

 

[bribri]

I'm using netatalk 4.2.0. Here's my afp.conf file:

[Global]
uam list = uams_guest.so
guest account = username
log level = default:info
log file = /opt/local/var/log/netatalk4.log

[Stuff]
volume name = Stuff
path = /Users/username/misc/afp
file perm = 0666
directory perm = 0777
ea = ad

The code is being edited with VSCode and compiled with Retro68, so it's already got LF line endings. All the files are ASCII too, so currently I'm side-stepping the encoding issue.

I want to see if I can track a CodeWarrior project in the repo just because I like the idea of keeping the codebase compatible with at least one classic mac compiler, even if I'm mainly targeting retro68. And yes, CodeWarrior can deal with LF line endings. At this time I'm not planning on targeting MPW or anything else that might require CR.

My goal is to be able to mount my codebase into a classic mac using AFP so that it can directly access the actual files on my modern mac that I'm working with, but still have a functional CodeWarrior project (along with any other classic mac files) that has its mac metadata and resource fork preserved in a way that git can track. (Not a big deal if they're opaque binary files. I'm never going to merge these.)

I thought about doing a conversion, perhaps using git hooks, but I just think it'd be much cleaner and easier if the files could be read and written in a way git can handle straight from the get go.

 

[bribri]

I just noticed some more unusual or buggy behavior. These are all happening on an emulated system in BasiliskII running System 7.5.3 with AppleShare 3.7.2 and Open Transport 1.1.2.

1. The AFP volumes I'm mounting list all files as being 27.9 MB.

2. The first time I mount a volume the extension mapping works and files have a correct type and creator code. However, when I close the Finder window containing these files and reopen it, their type and creator codes get reset to blank. If I examine the files on the host OS, I can see that they now have a "com.apple.FinderInfo" xattr. If I remove that xattr, then the extension mapping starts working again.

edit: The bug affects a real mac running System 7.5.5 as well.

 

[scj312]

@rdmark Unfortunately, I seem to be stuck in a restart loop once again:

I'll increase log levels so that next time we have something more useful to work with.

 

[rdmark]

I just noticed some more unusual or buggy behavior. These are all happening on an emulated system in BasiliskII running System 7.5.3 with AppleShare 3.7.2 and Open Transport 1.1.2.

1. The AFP volumes I'm mounting list all files as being 27.9 MB.

2. The first time I mount a volume the extension mapping works and files have a correct type and creator code. However, when I close the Finder window containing these files and reopen it, their type and creator codes get reset to blank. If I examine the files on the host OS, I can see that they now have a "com.apple.FinderInfo" xattr. If I remove that xattr, then the extension mapping starts working again.

edit: The bug affects a real mac running System 7.5.5 as well.

For number 1, enabling "legacy volume size" in afp.conf should make file size calculations work on older clients.

Number 2 is a bug I haven't encountered myself, so some more context would be needed to reproduce & debug. Can I ask you to file a bug ticket in GitHub?

 

[bribri]

For number 1, enabling "legacy volume size" in afp.conf should make file size calculations work on older clients.

Number 2 is a bug I haven't encountered myself, so some more context would be needed to reproduce & debug. Can I ask you to file a bug ticket in GitHub?

I tried adding "legacy volume size = yes" to my volume, and it had no effect.
Could there be a broader issue with certain settings not being used?

I created a GitHub issue for #2, and I think I found a way to consistently trigger it which I describe in the report.