VesperNet

> Intro
Here we are again, there has been quite a few updates since the project announcement back in 2025.
For those that know of the project since before, will probably remember MacDomain - that has retired from the public mainstream web and has since been rebranded as Arc.

In short, VesperNet provides, at the time of writing, a small alternative network / web ecosystem which is separated from the public mainstream web. It is powered by an alternative root DNS server, whatever domains / zones put into it is what becomes accessible. As such there are two websites to kick things off, found just below here.

VesperNet (.org)
Available both on the public, mainstream internet as well as behind the vesper network.
This website acts a little bit more like an informational hub / organizational website. It will tell you a little about the website and operations as well as basic setup instructions when it comes to the DNS itself, or why not try the HTTP proxy instead just for testing purposes? Either works, both are not required.

VesperNet (.net)
Available both on the public, mainstream internet as well as behind the vesper network.
This website acts as a service portal of sorts, which also allows creating a VN account (one account, many sites), which in turn can be extended with PPP options. Actual landlines are not required or supported.

Notable this year is that accounts do not need to be manually verified on any service, most features are applied automatically either on action or service request.

>> WIP Post
I am still in the process of moving (apartment to house) and cleaning (apartment), so this post will be edited / updated through March whenever I have enough time. There are many things not yet documented, but I believe this is as good a start as any.

> Why?
A genuine interest in creating and testing things that are mine, using / mixing old and modern technologies in ways they probably weren't meant to.

> Provided Services
To get started, there are some services on the network one should be aware of, at least when it comes to getting started with the basics - if desired. And there are quite a few to be discovered.

Web Directory
Not that different from the old DMOZ project, VesperNet is small so an actual web directory makes sense. Anonymous submissions are, of course, allowed.

Domain Registrar
This service is one of the cornerstones of the project, it is one of the few services that are not tied to the "global" VN accounts at this time. The service allows users to register and manage domains on the network. Packed with some neat features in an otherwise small package.

Web Hosting
Basic service to get users started if they wish to create websites and play with HTML and/or Gopher.
Provides a healthy 1.4MB of storage for that purpose, this is not a file storage / dump service. Users are also provided a subdomain as well as the option of linking their own domain registered via the domain registrar.

E-Mail
POP3, SMTP, Webmail - every registered user gets a @mail.com, with the possibility to add additional mailboxes via vanity domains. There are some useful features provided by this service.

Usenet
Unfederated and standalone, provides a basic set of newsgroups, easily expanded upon request.

Search Engine
This allows searching indexed websites on the network, there is a crawler that runs every now and then. The crawler honors robots.txt and respects the privacy set by site owners.

IRC / ICQ / AIM / Hotline
Connect in your lonesome, or gather some friends to talk to.
The Retro Chat Hub provides services and information.

> Notable Websites found on VesperNet
domains.com, search.com, webatlas.com, arc.net, news.net, mail.com, retrochat.net

The Tech
Nearly everything that powers and runs behind VesperNet created by myself has been written from the ground up, nearly everything created is with a purpose to make improvements between each projects. Bit of back and forth as new discoveries are made - so I'm 100% confident that there will be bugs, and incredibly happy when and or if these are reported.

So let's have a look at the ...

> DNS Server
Written in Python and stores zones as documents within a MongoDB collection, the latter can be a cluster or single server.
To prevent too much back and forth between the DNS and database server, I've also hooked the DNS server up to Redis which is able to cache zone data, this too can run as a cluster or single server. As frosting and sprinkles on the cake, there is also an in-memory response cache just in case, the two solutions honor TTL.

The DNS server is also ... autonomous, is that the right word? Anyways, it takes care of itself and adapts to load, does garbage collection, keeps track of malformed queries and protects itself from bots / scanners and a variety of different attacks. For stability, performance and privacy there is no logging.

While the DNS server is robust as a standalone server, it supports slave servers in different configuration constellations, or if the need arises, moves behind the curtains and stops serving requests to clients and only acts as a master to a master-slave server configuration to protect and uphold stability and performance.

Features
Master / Slave synchronization, UDP / TCP, DNS Records [SOA, A, CNAME, NS, MX, TXT, PTR, SRV], Reverse DNS, Zone Cache via Redis, Response Cache via in-memory storage, Negative Cache, DNS Compression, Private Network Gating (experimental feature), Self-Protection, Whitelisting, Blacklisting

> Proxy Sever
This is currently in the experimental stages, being written in Python with the purpose of being a quicker / easier way to test drive VesperNet. Unlike many proxy servers being written today, this one acts like an actual proxy server - it does not transform images from one format to another, it does not change https into http. It simply allows the source to connect to the destination as was intended.

Since the proxy server mostly tends to itself, it does not perform any logging, which is key to stability, performance and privacy.

Features
Performs lookups via the DNS server and automatically picks UDP with a TCP failover just in case, Caches known service IPs to prevent abuse (bots attempt to abuse open proxy servers), Bandwidth limiting and a few other nifty little features.

> PPP Server
Written in Python and C++, this is another way to get onto VesperNet (though without an actual landline). This feature requires an account set up via the VesperNet service portal as well as the official bridging software (GitHub Repo), or the Go version by @eric (GitHub Repo)

This is a fairly simple solution to allow users to use "dial-up" either from their emulated environments, or actual vintage hardware with the help of a serial cable. One could call it a tiny dumb VPN? Anyways, as a proof of concept connected users are free to communicate between one another, even point domains to whatever IP is assigned (if not static) and be "globally accessible", as demonstrated here. DNS Server magic (experimental feature).

There is also something being worked on for the PPP server, though not yet ready for prime time (sneaky peeky)

Apart from this, the server follows a strict no-logging model (stability, performance, privacy) where users connect to a director, which directs users to an available pool based on certain criteria from the director, which in turns handle the rest of the "magic". This solution allows pools to be located in different parts of the world, for lower latency, better / more stable connections.

Features
Advanced state tracking to maintain stable connections, custom PPPd build, different dial-up profiles (optimal connection, simulated real line conditions of the 80s/90s)

> Domain Registrar
Written in Python to manage zones, while this becomes super easy by working directly in the database, a web frontend will allow not only myself, but other users to register for an account and register / manage their own domains. The domain registrar communicates with the DNS server, so changes are near instant, the "propagation" time would be for however long the data is cached by the DNS server.

Features
Account registration / management, Domain registration / management, REST API for automated DNS management.

> Web Hosting
Ah yes, Python here as well, makes use of Redis and GridFS. The former for caching and the latter for storing user buckets.
Customizable subdomains, web based file manager to manage files and directories for web and Gopher, FTP access and Soon™ it should even provide WebDAV support. Domains via the domain registrar can also be pointed to the Arc server.

> Image Hosting
Way back when, this was a simple PHP solution that did nothing amazing, it has evolved over the years, been re-written from the ground up using Python, and is powered by SQLite amongst other things. The only thing retained is the web template itself.

It accepts most formats, but also the Macintosh PICT format (which conveniently gets converted to GIF) which was the main goal of the project from the very beginning. There is also support for user accounts, should one want to access and manage images from a private gallery. This project flies under VesperNet, is accessible both on the public mainstream internet and from behind VesperNet, some of you already know of Revontulet.

> E-Mail
You probably guessed it already, it's written in Python.
This consists of a POP3, SMTP, Relay Server as well as two web frontends (mail site, webmail). Previously mail could only be delivered within its own isolated little bubble, but it is now capable of sending to and receiving from other mail servers on VesperNet, provided they are configured correctly (mail server know-how)

And before anyone gets any wild ideas, yes, I've been meaning to implement IMAP support, but working on this server is making me somewhat furious

The mail service backend is powered by Python, Redis and MongoDB, and sports features such as vanity domains, multiple mailboxes (each domain is its own mailbox), catch-all for all domains, variety of authentication methods depending on how old the mail client is, multiple listening ports (in case your ISP does not entertaining the idea of allowing port 25 for instance)

> Usenet Server
Small, practical, Python.
The Usenet server is not federated, it does not pull or push data from other Usenet servers. Completely clean slate.
For practical reasons, the backends are MongoDB and Redis, and the server is meant for text, not binaries ... there is a web frontend which informs about how to connect and other nifty little things.

Features
User Groups, Permissions, Most Usenet server commands, simply Usenet.

> IRC Server
This one I've been tinkering with for quite some time, it could be done as a single server (which would of have made it super simple and easy), but I am a glutton for pain, so why take the easy route.

Written in Python, and relies heavily on Redis, even when standalone.
The server has the basic features one could expect from a basic IRC server (no services yet), connect, join a channel or two, chat with yourself or with friends, maybe even a stranger. Server linking is as easy as just spinning up more IRC servers, none of them actually connect or communicate with each other in a traditional sense, Redis handles that.

Features
User Modes, Channel Modes, "Server Linking", IRC Operators, Basic Commands, Self-healing / recovery

> Hotline Tracker
Slightly more advanced features that probably aren't found in other tracker software, written in Python and makes use of in-memory server tracking.

Features
Fake Server Listing Support, IP Blocking, Advanced Filtering, Optional Tracker Mirroring, Optional Admin API, Public API with optional key protection, Private Tracker Support, Optional indexing of news on all tracked servers, Optional indexing of files on all tracked servers, News/File list/content API

Some of the features found in the Tracker Server is used by and can be seen at BigRedH.

> Hotline Server
Others perhaps make use of Mobius for their HLServer needs, I'm not savvy enough to wave the wand of Go yet... so here we go, a modern Hotline Connect Server written in Python!

It comes with features such as full 64bit support to support files and folders over a certain size, as can be seen here (custom mobius client build), HOPE negotiation, full encryption support for Underline and GLoarbLine clients, Classic Mac resource fork support, file and folder comment support, folder upload / download support, REST API, configurable disk quotas (prevent reaching 0 bytes free), configurable download limits (global, per ip), configurable connections per ip, configurable bandwidth limits (global total, per ip), flat and threaded news, user accounts with extended protocol support (depends on clients), GIF icons, colored nicknames, public/private chat & chatrooms, web console and more.

Connecting clients can be identified to some degree, with custom server<>client behavior / feature availability depending on what the client supports.

Perhaps the most exciting or dumbest feature, depending on who you ask, is the file abstraction layer. The file area can be placed on a traditional filesystem, or why not a remote server with current support for GridFS and SeaweedFS.

> Archie Server
The Serial Port put down quite the effort in reviving Archie, though it didn't live very long. Ain't nobody got time to set up an emulated environment for something like this. Therefore, I took it upon myself to rewrite it using Python. It can be tested since it lives behind and outside of VesperNet.

There are some minor quirks left to solve, but overall this is an extremely performant working prototype that for all intents and purposes looks and works like the real thing... just faster.

Features
FTP crawler / indexer, Application / Web / Telnet access, V1 and V5 protocol support

> Search Engine / Web Crawler
Built with Python and somewhat of my first delve into the workings of how a search engine works, there's not a much magic behind it.
The crawler on the other hand (also Python) runs now and then as not to put pressure on site admins. It will ask the DNS server for a zone index, check each zone if port 80 and/or 443 is open for business, rebuild the index based on the results, check the HTTP response as well as look for a robots.txt, rebuild the index and begin crawling - depending on the response time it will adjust its own speed (with certain threshold as not to overload remote destination). Everything it comes across is parsed and tossed into the database.

Features
Click Tracking - completely anonymous as it helps ranking results in future searches, Basic search engine operands