eMate 300 / N2 Platform memory initialization

chuma

New Tinkerer
Sep 27, 2022
19
14
3
Canada
www.chuma.org
So the general question being asked here was "how does the system know how much DRAM is available?"

There is generally available documentation that leads to the answer.

First, we know how to add a new ROM Extension (REx) to patch the OS, thanks to the ROM Board Designer's Guide. RExes themselves contain multiple parts with different formats of patch data. In fact, the data on the eMate ROM board is a base ROM image with an Apple REx appended to it. The complete list of these is in the include file ROMExtension.h, available in the "Lantern" driver development kit plugin for MPW:

const ULong kRAMConfigTag = (ULong) 'ram '; const ULong kRAMAllocationTag = (ULong) 'ralc'; const ULong kDiagnosticsTag = (ULong) 'diag'; const ULong kCPatchTableTag = (ULong) 'jump'; const ULong kFrameExportTableTag = (ULong) 'fexp'; const ULong kFrameImportTableTag = (ULong) 'fimp'; const ULong kPackageListTag = (ULong) 'pkgl'; const ULong kPadBlockTag = (ULong) 'pad '; const ULong kPatchTablePageTableTag = (ULong) 'ptpt'; const ULong kGelatoPageTableTag = (ULong) 'glpt'; const ULong kROMTimingConfigTag = (ULong) 'romt'; const ULong kFlashTimingConfigTag = (ULong) 'fsht'; const ULong kFlashTimingBankTag = (ULong) 'fshb'; const ULong kFlashDriverConfigTag = (ULong) 'fshd'; const ULong kFlashDriverEntryTag = ULong ( 'fdrv' ); const ULong kFlashAddressTag = ULong ( 'flsa' );

Well, the first one in the list looks interesting :cool:

And kRAMConfigTag logically corresponds to the `RAMConfigTable` and related structs defined later on:
const ULong kRAMConfigTableVersion = 1; struct RAMConfigEntry { // From Includes:OS600:MemoryLanes.h ULong ramPhysStart; // physical start of Ram Bank ULong ramSize; // number of bytes of RAM in bank ULong tag; // tag of owner ULong laneSize; // size of a lane (e.g. 128K) ULong laneCount; // number of lanes (4 if entirely normal RAM) }; struct RAMConfigTable { ULong version; // version of this table format ULong count; // count of entries in the table RAMConfigEntry table[1]; };

Remember, the ROM image has a REx appended to it? Unfortunately it doesn't contain any parts tagged `'ram '`.

However, there's a good chance (I'm an optimist :LOL:) that there's at least one struct resembling RAMConfigEntry somewhere in the base ROM. If we found it, it would tell us what the parameters are for unmodified eMate hardware. It also means that an additional REx could be appended to the ROM that contains this data; it would take precedence over the base ROM's values.

And there are flash-based ROM boards, so a prototype memory expansion could be tested if these parameters needed to be modified in order for the system to address all available DRAM.
 
  • Love
Reactions: Kai Robinson

chuma

New Tinkerer
Sep 27, 2022
19
14
3
Canada
www.chuma.org
Thanks to the symbol information included with debug ROM images, and a convenient tool written years ago called `mpdumper` that demangles the symbols, we can look through much of the ROM code for interesting symbols. These are good starting points for tracing code and ROM accesses from Einstein's monitor.

There are these two data tables right at the start of the ROM, both of which contain known values for the start of physical RAM, flash banks, and hardware base addresses
* g8MegContinuousTableStart, offset 0x100
* g8MegContinuousTableStartFor4MbK, offset 0x174

And there are some interesting looking symbols representing function pointers:
* 0x0011E1AC ComputeRAMLimit
* 0x0011E318 InternalRAMInfo
* 0x0011E850 GetRAMSize
* 0x0011E854 TRAMTable::Init(SBankInfo *) static
* 0x0011E88C TRAMTable::Add(SBankInfo *, SBankInfo *) static
* 0x0011EC84 TRAMTable::GetRamSize(void) static
* 0x0011ED24 CopyRAMTableToKernelArea

These are all close to the code that reads REx parts, so they might be specific to the REx 'ram ' table after all. Some tracing with Einstein should verify that...
 

chuma

New Tinkerer
Sep 27, 2022
19
14
3
Canada
www.chuma.org
From tracing the boot process, there is code in the DiagBootStub - a very interesting area of the ROM almost certainly written in pure assembler - that tests whether addresses can be written and then read back, starting with 0x4000000 and 0x4200000 (base RAM and base RAM + 2MB). If the test at 0x4200000 is successful, then the routine is run again to test addresses 0x8000000 and 0x8200000. Seems like it's testing for memory banks ... need to see what will happen if the test at 0x8200000 is successful

Once there's a failure to read back the magic values written, there are writes to an unknown control register at 0x0F242400. Different values are written depending on the outcome of the two tests - values which are hard-coded in the data area of DiagBootStub. 🤔
 

Androda

TinkerDifferent Board Secretary 2023
Staff member
Sep 25, 2021
494
530
93
USA, Western
androda.work
DRAM chips on the eMate 300 motherboard are 2x TC51V4260. These are 256k x 16 chips, likely wired as two banks of 256k at 16 bits wide. This doesn't exactly explain the 900k of DRAM reported by the eMate in base configuration. My guess is that they are calculating it not based on bus width (512k RAM doesn't sound as good) but instead as how many 8 bit chunks there are. 512k at 16 bits wide can be thought of as 1024k at 8 bits wide, so the eMate has "1 megabyte" of memory. 1 meg of addresses at 8 bits wide if you consider bus high/low 8 as separate 'addresses'.

Here's a snippet of the CL-PS7110 datasheet. Note that this is *not* the same as the CL-PS7010 chip in the eMate, but it's the closest documentation I have.

1665058619625.png


The eMate's built in DRAM is 9x9 row/column configuration so there we see the address range of its available segment (and they also say it's 1 megabyte!). This doesn't exactly add up against what the ROM is doing, given that addresses here start at 0xC000.0000 instead of 0x4000000. But it's a different chip, so differences are probably to be expected.

At this point in the ROM where the memory tests are being executed, has the ARM710 MMU been instructed on how to remap memory? Or is the MMU set up as a result of these checks? I need to get a copy of the eMate ROM and throw it into something like Ghidra to poke around.
 

chuma

New Tinkerer
Sep 27, 2022
19
14
3
Canada
www.chuma.org
"eMate 300 (717006).rom" is the last file in the list under this directory of Apple Newton ROMs over at archive.org. Or, at Newton City.

Is that the ROM file you need, @Androda? :)
That's the correct-enough ROM image - it's named with the wrong version (it should be 737041) and it's not 100% clean, but it will do.

I started collecting information about actual clean NewtonOS ROM dumps, now that folks have hardware which can read the chips directly: https://github.com/MatthiasWM/NewtonROMReader/wiki/Clean-ROM-images
 

chuma

New Tinkerer
Sep 27, 2022
19
14
3
Canada
www.chuma.org
@Androda Yup, we're reading the same documentation. The PS-7110 is pretty close to the 7100 - the order of the internal registers between the two is very close.

At that point in the boot, the MMU has not been enabled.

I should actually go back and check my work ... for some reason I can't start Einstein with the eMate ROM, last time I used it I was trying to figure out how to get the boot to enter diagnostic mode and I fear I might have broken something :confused:

In any case, I will have copious time tomorrow to get my environment setup properly and dig into this some more.
 
  • Like
Reactions: Androda

chuma

New Tinkerer
Sep 27, 2022
19
14
3
Canada
www.chuma.org
How did you know that it is not 100% clean? :) I think many ROMs floating around are not clean, right?
Read the Clean-ROM-Images link that I posted. Essentially, dumping the ROM from NewtonOS reads the ROM space through the MMU, and there happens to be a page of RAM mapped in there. This is a minor consideration, but I'm a bit pedantic ...
 

chuma

New Tinkerer
Sep 27, 2022
19
14
3
Canada
www.chuma.org
I need to get a copy of the eMate ROM and throw it into something like Ghidra to poke around.
Ah, yes, that's a good idea! I started doing that last night, though I've never used Ghidra before. I am not great at reading machine code (it's been a long time since I had to use it in university, and haven't had to use it in my career) so it's nice to get some kind of visual representation of things.

If you're doing this and want some to import some symbols, there's tool that I collected a long time ago called mpdumper that dumps and demangles the symbols from the debug Newton ROM Images from Apple's old DDK, which are in AIF format: http://www.unna.org/view.php?/development/tools/ROMSymbolDumper

These debug ROM images are on UNNA: http://www.unna.org/view.php?/development/Debugger_Images

Last night I modified mpdumper's output, further processed it with some sed into a form that I could import into Ghidra using the included ImportSymbolsScript.py. It's not perfect, but most data symbols are marked as labels, and most C/C++ functions are marked as such. Here's DiagBootStub showing the two calls to the (as yet unnamed) subroutine that tests memory areas:
ghidra-emate.png
 
  • Like
Reactions: retr01

Androda

TinkerDifferent Board Secretary 2023
Staff member
Sep 25, 2021
494
530
93
USA, Western
androda.work
It's not perfect...
Wow, this is great information. I've never used any of these tools before (other than Ghidra) so it might take me a bit to get set up and understand them. It also sounds like we'll need a dev rom board to try and increase these settings, I would guess that ROM patches can't do it?
 
  • Like
Reactions: retr01

chuma

New Tinkerer
Sep 27, 2022
19
14
3
Canada
www.chuma.org
As far as I can tell, Einstein currently doesn't boot with the eMate ROM - it ends up trying to run the diagnostic image from REx0 and falls into a boot loop. Could be because Einstein applies a number of ROM patches in its JIT emulation to work around things that aren't understood or hardware that doesn't exist, and the location of the patches in the eMate ROM hasn't been implemented. However the low-level boot code of the eMate and the MP2100 ROMs are practically identical, so I've been tracing execution of the 2100 ROM in Einstein while following along with the eMate ROM in Ghidra, and it's not a problem.

Actually I spent most of my afternoon boot tracing (the 2100 ROM), mucking with Einstein, and commenting functions in Ghidra. Found some strange results, too.

When simply increasing the RAM size to 8 MB in Einstein, NewtonOS boots (eventually*) but reports ~16MB of RAM 🤔 This has all the RAM starting at 0x04000000. But when I modified Einstein to allocate 4MB at 0x04000000 and another 4MB at 0x08000000 - which also involved making the second memory config register return the same values as the first - then NewtonOS reported 8MB.

* and every time I boot with 8MB, the boot code enters a function at 0x19x10 that seems to be ... calibrating RAM timing or something? It takes a long time to complete, to the point where I thought the emulator was boot looping, but it eventually made it to NewtonOS.
 

chuma

New Tinkerer
Sep 27, 2022
19
14
3
Canada
www.chuma.org
I haven't posted any updates here because the Linux PulseAudio output driver for Einstein was killing the emulation performance. Since I hacked/wrote it, I just spent the last 1.5 weeks fixing it. Now that it no longer sucks and the emulation actually runs smoothly and at the proper speed on my Linux workstation, I'll get back to this soonish - as long as I don't get distracted by anything else...
 
  • Like
Reactions: splorp