HTTP proxy and transcoder for vintage browsers

rdmark

rdmark

Moderator
Staff member
Oct 3, 2021
116
176
43
Do you all have experience running HTTP proxy and transcoders to get modern web content into a shape that vintage web browsers can handle?
After some cursory research, I found two active open source projects: WebOne and Browservice.

The latter uses full on backend rendering, and passing images of the rendered pages to the user agent, which is not quite what I was looking for.

The former seems more promising, as it does selective transcoding of web contents, and decryption of HTTPS traffic, for rendering in the vintage browser. From some cursory testing, it seems to work fairly well for text-centric content pages for rendering on a mid-range Power Mac type machine, in particular when enabling low quality jpeg re-compression of image files. However, overall it is not quite aggressive enough in transcoding html/css/javascript contents to be really useful for general browsing on, say, a 68040 based Mac.

Are there other solutions out there that are more suited for early 90s, or earlier, systems?

I've been looking into this as another potential companion utility for RaSCSI, following the integration of Netatalk for AppleShare file sharing in the 21.10 release. It's all well and good to emulate an Ethernet adapter to get old machines online, but having a convenient transcoder running in parallel would take it to the next level I think. :)
 
rdmark

rdmark

Moderator
Staff member
Oct 3, 2021
116
176
43
So I've ran a few tests, and macproxy is close to being an excellent solution for very low-end machines, but it has two major drawbacks:

For one, unlike WebOne is does not translate https requests to http, which means that any inline hyperlink on an https site will fail. You can still copy paste that URL into the address bar and change the protocol to http to get by this limitation, but it's a huge inconvenience.

And secondly, it's a Python2 script, which is no surprise given that the last commit was 8 years ago. But it means that running it on a modern system has additional overhead, and brings risk of dependency nightmare.

Both shortcomings are probably relatively easily overcome with some time investment, but in the meantime I'll keep researching more recent/active projects. :)
 
rdmark

rdmark

Moderator
Staff member
Oct 3, 2021
116
176
43
There, I forked and improved the macproxy code a tiny bit to make it run with Python3 and venv (actually, very few changes had to be made.)

github.com

GitHub - rdmark/macproxy: A simple HTTP proxy script for putting old computers on the Web

A simple HTTP proxy script for putting old computers on the Web - rdmark/macproxy
github.com github.com

It's not consistently successful in translating all https requests to http. Some sites work fine, but others like arstechnica.com don't, and you have to manually change the protocol in the address bar after the error. Somebody who has actual knowledge about Flask and requests can probably do better here. :)

Anyhow, I really really like how light-weight and lighting fast this proxy is. When it works, it makes it a breeze to browse the web on 68k Macs.
 
  • Like
Reactions: alxlab
rdmark

rdmark

Moderator
Staff member
Oct 3, 2021
116
176
43
There, I pushed a few changes to my fork so that inline hyperlink URLs should now always be rewritten to the http protocol. It was just a matter of using beautifulsoup to modify the html in memory.

I think this proxy is eminently usable at this point. The one major improvement I can think of now would be image transcoding to low quality jpeg and/or indexed gif.
 
alxlab

alxlab

Active Tinkerer
Sep 23, 2021
287
305
63
www.alxlab.com
Yeah not sure how that could be done transparently.

Off the top of my head I could see the proxy downloading the image, converting them with image magick or the like, hosting it locally in directory path named to uniquely identify the page and image for current/future use and then replace/inject the new urls into the page.
 
rdmark

rdmark

Moderator
Staff member
Oct 3, 2021
116
176
43
Figured out the easy way to get embedded images working! I tweaked the proxy logic to only modify the contents when content-type is 'text/html'. This way, any traditional embedded image (<img src="">) in traditional formats (jpeg, gig) work absolutely swimmingly on a higher-spec 68k Mac (Performa 476 below).

It slows down page loads in MacWeb on my Mac SE a bit since it has to handle a bunch of "image load errors" but still perfectly usable.

The problem to tackle is binary file downloading. It works fine if you 'save as' with the latest change, since the proxy no longer tries to mess with non-html contents, but I need to tweak the response headers a bit for the user agent to recognize it as a binary stream, I think.

IMG_4022.jpg
 
rdmark

rdmark

Moderator
Staff member
Oct 3, 2021
116
176
43
Binary file downloading is now working as well, at least in Netscape 4.5 on Mac. I had to use a workaround with Flask sessions to retain the content-type header, which Flask always changes to text/html regardless of the original header. There may be some decorator or hook or something to achieve the same thing more elegantly, but so far it seems to work great.
 
rdmark

rdmark

Moderator
Staff member
Oct 3, 2021
116
176
43
After a bit more tinkering, making the port number configurable and cleaning up the code, I cut something I'd like to call a stable release of macproxy (off of my fork of course; I hope the original author don't mind) here:

github.com

Release Macproxy v21.11 · rdmark/macproxy

Macproxy v21.11 Changes since the original Macproxy codebase by tghw: New features Configurable port number Made embedded image loading work. (Pinpoint html transcoding to 'text/html' only.) Made ...
github.com github.com

If you want to use this with a really old machine, such as a 68000 Mac, you'd want to turn off image loading in your browser (e.g. MacWeb). But overall, I consider this an amazingly usable little proxy server for most casual web browsing on a vintage computer. Please try it out!
 
  • Like
  • Love
Reactions: davewongillies, Byte Knight, Daniel Hansen and 2 others
rdmark

rdmark

Moderator
Staff member
Oct 3, 2021
116
176
43
Bumping this thread to mention that I cut another three releases in December that improves compatibility with the modern web, and introduces a few advanced command line options to further customize the proxy.

The most important change is that it no longer mangles the layout of www.frogfind.com ;)

See the full changelog at: https://github.com/rdmark/macproxy/releases
 
Byte Knight

Byte Knight

Tinkerer
Oct 21, 2021
90
72
18
Thanks for the updates! I haven't tried it out yet - what advantages does this have over just using frogfind.com which also converts modern web pages?
 
rdmark

rdmark

Moderator
Staff member
Oct 3, 2021
116
176
43
@Byte Knight That is a great question! My executive summary would be something like:
  • Macproxy is more compatible with the modern web and better at retaining and displaying valuable content, but the tradeoff is that you will often see a bunch of ugly looking headers and footers (e.g. exploded site navigation toolbars and rails) that take up valuable space.
  • FrogFind (which leverages the readability.php library) is much more aggressive with the downsizing of contents making it more readable on small screens, but overall less compatible with the modern web (you may have seen the "Invalid or incomplete HTML" error) and often remove valuable content.
It's well worth noting that the two aren't mutually exclusive. You can have Macproxy running and proxying requests, not affecting the appearance or functionality of frogfind.com since it doesn't have any imported resources that get decomposed by the proxy.

A few other benefits of Macproxy over FrogFind:
  • It leaves 'vintage' websites intact
  • Inline embedded image support
  • Binary download support
 
Last edited:
  • Like
Reactions: Byte Knight, JDW and rjkucia
You must log in or register to reply here.
Top Bottom