New (small) build

Howdy.

After a long break I present you herewith a new test iso for Efika. It's much much smaller than all previous images, because I have not included the "contrib" build. The reason was instability of some third party MUI classes which either crashed the Zune preferences or did not allowed the prefs program to start.

The most important changes worth mentioning are USB fix, sync with current source tree and extended debug. The first fix repairs an anciend bug in OHCI driver, where the USB ports were powered up upon driver's initialization. It lead to some "zombie" devices on the bus. The Efika source tree has been synced with the SVN repository too, therefore Efika build got all recent changes and updates from the main tree. The most important and Efika specific is extended debug. Until now, if anything on native aros crashed, I have had no clear information about the code that failed. Just some address in memory. Debugging under such conditions was so ineffective, that I've finally decided to improve the crash log a bit.

Now, every single module in the system registers itself at the kernel.resource with name and headers of ELF file. Kernel resource scans the file and remembers all symbols referring executable sections. The same is done with Kernel during the bootup too. Now, if a crash occurs, I become much more verbose information, which eventually helps :) The information includes the location of crash (which byte offset, which function, which module) and the call backtrace up to the birth of the task/process. Have a look at NList crash:

[KRN] Exception 3 (DSI) handler. Context @ 0xff7fb948, SysBase @ 0x32c0, KernelBase @ 0x4bac
[KRN] Process 0xcae00 (Zune)
[KRN] Crash at byte 8 in func strcmp, module muimaster.library
[KRN] SRR0=00ed4428, SRR1=0000f030
[KRN] CTR=ff8066e4 LR=00e94270 XER=00000000 CCR=28004044
[KRN] DAR=00000013 DSISR=40000000
[KRN] HASH1=07000000 HASH2=070fffc0 IMISS=00ed4420 DMISS=00000013
ICMP=80000003 DCMP=80000000
[KRN] SPRG0=ff7fbaf0 SPRG1=00da9598 SPRG2=28004044 SPRG3=00000000
SPRG4=00004bac SPRG5=000032c0
[KRN] GPR00=00e80980 GPR01=002c9950 GPR02=00000000 GPR03=00da9598
[KRN] GPR04=00000013 GPR05=00da50c0 GPR06=002c99b8 GPR07=00da9598
[KRN] GPR08=012ecc00 GPR09=0000004e GPR10=00000120 GPR11=00da50c0
[KRN] GPR12=48004022 GPR13=00000000 GPR14=00000000 GPR15=000d0000
[KRN] GPR16=00f5fed0 GPR17=00000001 GPR18=00f60000 GPR19=011cf4bc
[KRN] GPR20=00000002 GPR21=00000000 GPR22=00000000 GPR23=00f60000
[KRN] GPR24=00db0000 GPR25=00da5120 GPR26=00000000 GPR27=8042ac64
[KRN] GPR28=00da50c0 GPR29=00f19cd0 GPR30=00000013 GPR31=00d87620
[KRN] Hash1 dump:
[KRN] 80000010.04000012 800007a0.f800f03a 80000400.8000803a
80000410.8400803a
[KRN] 80000420.8800803a 00000000.00000000 00000000.00000000
00000000.00000000
[KRN] Hash2 dump:
[KRN] 8000000f.03fff012 800007bf.07ff0010 8000040f.83ff703a
8000041f.87ff703a
[KRN] 00000000.00000000 00000000.00000000 00000000.00000000
00000000.00000000
[KRN] Instruction dump:
[KRN] 00ed4428: 88040000
[KRN] 00ed442c: 7c604851
[KRN] 00ed4430: 4c820020
[KRN] 00ed4434: 2f890000
[KRN] 00ed4438: 4d9e0020
[KRN] 00ed443c: 39000000
[KRN] 00ed4440: 7d674214
[KRN] 00ed4444: 7d444214
[KRN] Backtrace:
[KRN] 00e80980: byte 40 in func MUIMaster_MUI_GetClass, module
muimaster.library
[KRN] 00e82070: byte 56 in func MUIMaster_MUI_NewObjectA, module
muimaster.library
[KRN] 011049d0: byte 176 in func MUI_NewObject, module NList.mcc
[KRN] 010d9a50: byte 632 in func mNL_New, module NList.mcc
[KRN] 010db7e0: byte 880 in func _Dispatcher, module NList.mcc
[KRN] 00e941cc: byte 28 in func metaDispatcher, module muimaster.library
[KRN] 010d35c0: byte 48 in func DoSuperMethodA, module Mailtext.mcc
[KRN] 010d291c: byte 80 in func Mailtext__OM_NEW, module Mailtext.mcc
[KRN] 010ceac0: byte 204 in func Mailtext_Dispatcher, module Mailtext.mcc
[KRN] 00e941cc: byte 28 in func metaDispatcher, module muimaster.library
[KRN] ff8af35c: byte 40 in func CoerceMethodA, module intuition.library
[KRN] ff8842dc: byte 164 in func Intuition_NewObjectA, module intuition.library
[KRN] 00e82098: byte 96 in func MUIMaster_MUI_NewObjectA, module
muimaster.library
[KRN] 0110b27c: byte 176 in func MUI_NewObject, module Mailtext.mcp
[KRN] 01108088: byte 952 in func MailtextP__OM_NEW, module Mailtext.mcp
[KRN] 01105b68: byte 180 in func MailtextP_Dispatcher, module Mailtext.mcp
[KRN] 00e941cc: byte 28 in func metaDispatcher, module muimaster.library
[KRN] ff8af35c: byte 40 in func CoerceMethodA, module intuition.library
[KRN] ff8842dc: byte 164 in func Intuition_NewObjectA, module
intuition.library
[KRN] 00e01110: byte 2476 in func init_gui, module Zune
[KRN] 00e016f8: byte 276 in func main, module Zune
[KRN] 00dffcb4: byte 452 in module Zune
[KRN] ff8c2a38: byte 36 in func DosEntry, module dos.library
[KRN] ff8c5274: byte 13092 in module dos.library
[KRN] **UNHANDLED EXCEPTION** stopping here...

The next release will get rid of the "stopping here..." message. Instead of halting whole aros, the failing task will be moved away, a popup message will appear, and AROS will continute to work. Stay tuned...

One tiny fix...

... and suddenly everything goes better :-)

as you may remember, I have released a test pre-alpha iso with AROS for Efika recently. However, there were some issues with using it. Some people tried to boot it from DVD-USB devices, some tried USB Flash Keys. The former succeeded mostly (but not always) whereas the latter failed pathetically. I was scratching my head and found no sane solution. Once, being completely desperate, I have enabled full debug of OHCI driver, in order to see every single transfer description used by all USB devices in the system. There was it! In some cases, OHCI chip was instructed to read 512 bytes starting from physical address 0xffffffff. That cannot be and it couldn't work. At that point the reason of OHCI hanging was known. But why was it like this???

Well, the answer surprised me a bit. The issue was the sign extension during APTR to uint64_t cast. At some moment, AROS wanted to read from all bootable devices the first sector into it's local table in .bss section. Since that piece of code belongs to kernel, the virtual address of this portion of memory was somewhere around 0xff7f0000. Sign extended address, 0xffffffffff7f0000 was of course not found in the MMU hash table and the KrnVirtualToPhysical() call failed. Sign expansion fixed, et voilà! All the issues with mass storage are gone.

For those who do not own an Efika, a tiny screenshot:


here, you can see the PCI Tool (doesn't show much because the most of Efika devices are on MPC5200B SoC), the screenshot tool itself, About AROS window with the mysterious "chrp-ppc Build efika", and shell. I wanted to show you there, where the SYS: assign is pointing to. The volume labeled AROS: is a boot device named UH0. This is the USB flash key. The USB1: device is my DVD drive connected throughout USB bus.

The torrent file is here, if anyone of you is willing to test it. Keep in mind I have toyed with scheduler there. If it does not work smooth for you, let me know. If it works well, let me know too. If the iso fails on your machine, do not hesitate writing me about (in such case try to provide the debug log, if you can).

PS. If the boot starts before the USB key is enumerated, it will not be bootable for AROS. It sucks, but that's the way AROS boots (some people are considering few changes there). In such case, you may add a command line option, eg. bootdelay=5 will introduce additional delay of 5 seconds right before booting. The iso has a bootdelay of 3 seconds by default, because all my USB devices attached to efika (keyboard, mouse, flash key and DVD) are sitting on a hub.

New efika iso

The last ISO you got was a bit annoying. There were almost forty modules to load by OF. It took long. Moreover, putting these files on FAT-formatted USB stick failed, since OF was unable to find some of the files.

I made few changes to the bootstrap code and AROS itself. AROS sets up interrupts in proper way. It should not lock anymore, as it happened before. The USB mass storage has been fixed a bit and should work a bit better now. Regarding the bootstrap code, following changes have been made:

• os_image does not need the boot-device setting anymore. Boot it from any place and it will work as expected.
• os_image honors the command line argument passed from OF. Now, if both menu.lst and OF command line contain parameters for kernel, they will be joined together.
• os_image supports packages. No need to load 40 boot files through OF anymore. There will be few packages: generic-ppc, efika, usb and few other files to load. It means faster boot time and less risk of OF issues ;)
• os_image does not support timeout option anymore. If there are more kernel configurations, it will stop and wait for an input.
  

Although timeout is gone, os_image on this cd will boot aros without waiting. I have removed the second boot option, which was not used anymore. How to boot? Put the CD into drive and type:

boot cd os_image

whereas "cd" corresponds to the boot device you use. You may find this iso here.

Updates, updates

OMG, it was a busy weekend... :)

Last 48 hours were quite intensive. The source tree of Efika5k2 branch has been updated - the tree was almost 4 months old! I have added a complete contrib directory to the build - the iso is now 150 MB large and contains a complete gcc compiler toolchain for Efika as well as some other goodies.

The kernel.resource got new function, which may be used to change the WMID bits of certain pages of memory. One may use it, for example, to disable caching in some RAM areas. This feature is used by the PCI driver class for efika. Every region allocated for use by PCI bus master devices is excluded from caching. This manouver may simplify driver's code significantly. For example, the OHCI driver uses Endpoint Descriptors (ED) and Transfer Descriptors (TD) which may be used by both CPU and OHCI controller. With an incoherent cache enabled, the driver has to either flush or invalidate the cached copies of descriptors permanently. Here, uncached region for these descriptors permits a more simple and error-safer code.

I have changed the scheduler a bit. The old one was not so stable upon very small CPU loads. It's tendency was to lock interrupts and put CPU into sleep mode. Very unfortunate, I must say ;) The updated code introduces the Idle Task which enters the power saving mode in a safer way. The scheduler calculates the time which CPU spent during execution of very task. Have a look yourself at C:TaskList command to understand what I mean :)

Wait, Did I wrote "have a look yourself"? Yes! Download this torrent, and test the AROS for efika yourself. It is not complete yet. You will surely see it crashing. The both IDE and network drivers are missnig. The release is unstable. Just put the ISO on CD, put it into your USB-CD or USB-DVD drive and start efika. Then, type following commands in the OF prompt:

setenv boot-device cd
boot cd os_image

Once you see the Boot? > prompt, press 1 and be patient. The OF needs some time to read all aros' modules and will put enourmous amount of debug info. It will even report some missing file. Once AROS starts up, you will see the mouse pointer on the screen and CD drive working. Have fun. The serial port outputs debug information at 115200bps, 8N1.

PS. My upstream is limited. Please seed the file after download.

Coincidence, or MassStorage contd.

Every modern USB presents itself as a SCSI conforming bulk-only mass storage device to the system. Therefore, in order to do any IO operations, I have had to do them in the old good SCSI style. Therefore, the mass storage class do contain a DirectSCSI method and exposes it to rest of the world.

What a coincidence! I have added a few-lines long implementation of HD_SCSICMD to the .device layer of mass storage. I have started AROS in QEMU and forwarded the USB->PATA converter with a DVD attached to it and AROS CD inside (I didn't want to test anything special, but it was the first CD I saw on the desktop). AROS booted and stopped. Then, the timeout errors appeared. A lot of them. A bit disappointed I've left my desk and did something else. Suddenly, the CD started to spin and AROS booted from USB CD device! Great!

A short investigation showed, that the CD filesystem used in AROS performs the INQUIRY SCSI command with a fixed size, whereas the USB protocol provides a much shorter data in return, which leads to a timeout. Then, it tries to get the full-sized TOC and such. Anyway, AROS team will either need to fix CD filesystem a bit, or I will have to give up 10 second timeouts in mass storage and introduce four pipes there: two of them with very short timeout (100 miliseconds or so) and two with the 10 second timeout.

So, now the mass storage can not only access any USB sticks and USB harddrives, but also USB CD/DVD readers and burners. Nice, isn't it? :) The next step: booting Efika CD directly :)

Back to Efika

Good Day Everyone!

After my last fights with the USB Mass Storage bounty I have decided to take a break there and continue with AROS port for Efika. The Mass Storage is basically ready, at least in my eyes, since I do lack the hardware where the whole USB stack makes any issues at all. It works here with two PC's I own and with two virtual machines: QEMU and VmWare Player.

Anyway, I'm back with Efika right now. I took my freshly written mass storage class, fixed the OHCI driver again (it uses the physical address provided by the CachePreDMA() call) and copied the whole Efika CD onto my SFS partition of USB stick. And guess what! I have the wanderer desktop on Efika in front of my eyes now! Horray! Screenshot will follow as soon as I organise a USB hub (two USB slots are not enough for mouse, keyboard and USB stick, you know :)

One note: as you may know from the lecture of my previous posts, both FAT and FFS filesystems lack proper caching and thus are relatively slow, especially on Efika. Attempting to boot from FFS partition on USB stick may take more than 10 minutes. Booting from FAT will not work, because this filesystem is still incomplete. As for SFS, Efika needed one minute to boot from it, approximately.

PS. Since the SFS filesystem for linux is not working well on my machine (it crashes the linux kernel during mount already), I have started VMWare with AROS and Efika cd's attached. Then I have plugged my usb stick, attached it to vmware and copied all the Efika ISO contents to the stick. Just as a proof that all the things are working here as expected.

Almost there :)

This week was a very active one. I was working on my mass storage class all the time and got some really impressive results. The bounty is almost ready, at the moment I'm testing the class and the USB stack as a whole too. I have tested the mass storage class on my two USB sticks and a 4-in-1 USB card reader. All of them seems to work properly. In my private test phase were incorporated three different machines:

• my dying laptop (either gfx chip or it's VRAM attempts to die pathetically - a chance of successful boot into X11 or Windows oscillates around 5%) used for development
• x86_64 box with 32 bit native aros to do some heavy testing
• my wifes PC which was the very first computer in the world booting AROS from USB stick ;)

Additionally, a qemu with USB forwarding has been used for immediate tests (in this case, the qemu takes a whole device of my wish over). In all this cases the USB stack and mass storage class seem to work properly. The mass storage is also being tested by other people, whom I'm very thankful :)

Once the eventual bugs are ironed out the bounty, according to it's description, will be completed.

Note about the speed: AROS' USB stack has at the moment the OHCI and UHCI drivers. It means, only the USB1.1 speed may be expected. The closest to the maximal transfer rate is, with 1.4MB/s, the SmartFilesystem. Both fat and ffs filesystems are dramatically slow, achieving 160KB/s and 140KB/s, respectively. Keep this information in minde while toying with AROS :)