Done (almost) with the bootloader

Hello again!
Last time, I said that there were almost no differences between the IAP loader drafts 2 and 3. Well, turns out I was wrong. Just to remind you, draft 2 specifies that the program should be divided into two parts - one which contains the IAP loader and cannot be overwritten and the second one that contains the rest of the code and can be overwritten freely. Draft 3 adds the ability for the IAP loader to copy itself into RAM and run from there in order to allow it to rewrite itself in FLASH.

Problems

When I started work on the draft 3 I ran into two main problems. The first one, which I sort of expected, was that the IAP loader, together with all the functions it uses, can only barely fit into the RAM. I managed to squeeze it in by trimming some of the code, but I expect that there would be trouble if we, for example, wanted to allocate memory for some large buffer in some later version of the firmware. The second problem was more severe and, in the end, it made me abandon the idea of implementing draft 3 altogether. The trouble is that the micro often locks up in an error state when there is a transition from the code residing in FLASH to the code which is in RAM and vice-versa. I don't know what exactly causes this and it's probably not an unsolvable problem, but without a JTAG debugger and with my limited experience, trying to fix this would most likely take way too much time. The advantages that draft 3 has over the draft 2 are probably also not worth it, especially considering the approaching deadline on my project. Therefore, I decided to switch to working on implementing draft 2.

Working on draft 2

Implementing this was actually quite simple. It was only necessary to modify the ldscript a little bit by including a new output section called .boot containing the IAP loader and all of the other functions it requires to work properly. These were tagged with __attribute__ ((section (".src")))  so that the linker can place them in the appropriate section. I also modified the write and erase IAP functions to prevent the user from overwriting the part of the FLASH memory containing the IAP loader. The .boot section is aligned to 1kB boundary so that it will span a whole sector or multiple whole sectors of FLASH because it can be erased only in a sector-sized blocks.
The only problem I had while implementing this draft was with a constant reference table (or array, if you wish) which is used to speed up the CRC calculations. In the original code it was declared locally inside the CRC computing function and the linker somehow placed it in the rewritable part of the code. This caused the micro to freeze every time I tried to reprogram it. The solution was to declare the array globally and tag it with an attribute specifying that it should be placed together with the IAP code. I should keep this in mind in case I run into a similar problem sometime later.

In conclusion

Our IAP loader occupies two first sectors of the FLASH (so there are six left) and is protected from being overwritten. This, I think, is a fairly standard solution and how most other boot-loaders work and I am quite content with it. It is not really necessary to have the option of reprogramming the IAP loader itself right now and implementing this feature would probably be more trouble than it's worth. Right now, I can't say that the IAP loader works 100% yet. To do that I will have to test it properly first.

Until next week...

Bootloader practice

Good day again!
Yesterday I wrote about how the current boot-loader operates. Today I would like to write about what I have actually accomplished while working on the new boot-loader.

Draft no.1

First I tried creating a boot-loader that would operate as per the draft no.1 described in the previous post. That is - the program is stored in FLASH in its entirety but right after reset everything gets copied into RAM and is executed from there.
At first I was really fumbling around while trying to accomplish this, but then I found a tutorial (accidentally also written here on Blogger as I have just noticed) which describes basically exactly what I need. The only difference is it is written for a different board than LPC1343, but that's not a huge problem.
By following the instructions from the tutorial I finally managed to succeed. It required a complete rewrite of Crt0.S and ldscripts. There are some little tricks involved in how it all works but I won't describe them here as that would be redundant as they are covered sufficiently in the tutorial.
As I mentioned in the previous post, the program proposed in this draft is not really practical as there is no way the whole firmware will fit in RAM. However, it is useful for testing and presented a nice way to get some more experience before moving onwards to the successive drafts.

A little bit of semantics

Firstly, before I continue writing about the drafts, I think that it would be appropriate to clear up one particular semantic issue that might be confusing to the reader. My ultimate goal is to create a new boot-loader, but a one that will not replace the current boot-loader that is already present. In fact when I am done there will be two sub-programs that one could call a boot-loader - one that is responsible for initialising the micro after reset and which is the currently present one and a second one which is the one I am trying to create and which will allow the micro to be reprogrammed over the RS485 bus. To make things more clear I will refer, from here on, to the first boot-loader as reset-loader and to the second one as IAP-loader.

Drafts 2 & 3

Next up, I started to work on creating an IAP-loader that would work as our draft no.2 specifies. During the actual work I found out that the differences between the no.2 and 3 drafts are, in fact, minimal so I have decided to roll them both up into one project. The only difference between them is that in draft no.2 the actual IAP-loader part cannot be rewritten and can be placed in either FLASH or ROM, when in the no.3 draft the IAP-loader part has the ability to overwrite itself in FLASH and therefore has to be placed in ROM. This can be changed by rewriting about 3 lines of code, hence the justification for merging them both together.
I have also found out that, in the end, the existiong Crt0.S and ldscript files had to be only slightly modified to acheive the desired functionality. Basically it was only necessary to add an extra section that contains all of the IAP-loader code, into the ldscript, link it for RAM, store it in FLASH and extend the copy loop in the reset-loader to copy it all on start-up. Actually it is not even necessary to modify the reset-loader code and we can just do the copying on runtime from the main function.

Problems

So you might ask: If everything was so easy, what took you so long? Well, as usual, I ran into a lot of little problems which slowed me down. One in particular, I fell I have to describe here:
I kept getting usage faults from the processor when it was supposed to execute a jump from the code in FLASH to the IAP-loader code in RAM (basically just calling the IAP-loader function). In the end all that had to be done to solve it was to change the order of the sections contained in the ldscript. When the section containing the IAP-loader code (I labelled it as .boot) is placed before .text I keep getting faults. If it is the other way around, everything works fine. I have no idea why the micro behaves this way; I am just glad it works properly now.

Future concerns

While working on the no.2 and no. 3 drafts I didn't use an actual IAP-loader code which would be able to reprogram the FLASH memory. In order to do that, I would need to have the new version of tolpc extended with the IAP programming functionality available on the master computer. Sadly, it is still not availiable. Mr Svoboda promised to code it but lately he has been busy with other projects and hasn't had the time to do it yet.
In place of the actual IAP-loader I have, therefore, used only a simple placeholder program which only blinked with the LEDs and sent some test messages over the serial interace.
When I will get around to implementing the actual IAP-loader, I will have to solve one specific problem - What to do with the parts of code that are common between IAP-loader and the rest of the firmware? The simples solution would be to just place it all, together with the IAP-loader, into RAM on runtime. That is ,however, provided that it will all fit in there together with all of the buffers, arrays and other variables. If not, I foresee a lot of work with function pointers and memory overlays and such when I will have to figure out how to make it all fit somehow. We will not find out how it goes until we try it though.

Measuring the current

I have also been encouraged to start working on the following task on my list, which is upgrading the current firmware to enable it to measure the current running through the four coils which are present on each MagMan module. Everything on the hardware side is already, wired and prepared for this; It is just that nobody has yet had the time to program it.
So far I have only done some reading on how the ADC works but I can already tell that, while implementing this feature, I wil have to make a lot of decisions about how it's all going to operate. For example:

• What sampling frequency to use?
• Whether to use stroboscopic sampling.
• Define new Elvis messages to control the sampling process and to read the data from the modules.
• Decide how often to read the data and how to store it in between the reads.
• Whether to do any pre-processing on the data.

So, as always, there is not a shortage of work for me.
Write to you next week!

Bootloader theory

Hello,
I have just realised that I didn't write any update last week, but I wouldn't have had much to write anyway as there was little progress.
I have been busy working on the boot-loader. It consisted mostly of trying different things and failing. At least I have become more familiar with writing linker scripts, assembly and using binutils.

The current bootloader

The current firmware is (of course) not without boot-loader. My task now is to extend the existing boot-loader and firmware to allow for programming over the RS485 bus. I may have alluded to how the current boot-loader operates in the previous posts, but I think it will be useful if I describe it here again in a coherent fashion, so that everybody will be able to at least somewhat understand what I am talking about and the changes I made.
Firstly I have to make yet another detour and explain a bit about how the linker works and why do we need it.

Linker 101

Your typical program consists of several things: Firstly, the code itself - machine instructions for the processor. Parts of the program containing these are labelled as .text in the object files. Then there are initialised and uninitialised variables, which are labelled as .data and .bss respectively. Constants are labelled as .rodata.
What linker does (described in very primitive terms) is to place these different parts of a program where you tell it to in the memory and set the symbolic links between these different parts according to this placement so that it all works together. The thing is, that we want our program to be placed in FLASH because it's non-volatile but we need our variables to be in RAM during runtime so that it will be possible to change their values. One more thing that has to be done when working with Cortex-M3 is to place a valid interrupt vector table at the address 0 (bottom of FLASH).
Our linker script therefore does the following:

• Places the interrupt vector table (specially labelled as .ivec) at the FLASH bottom.
• Places .text together with .rodata into FLASH. We don't really care where, but if not specified otherwise the linker will just put it right after .ivec.
• Now comes the interesting part. The section .data containing the initialised variables has to be loaded into FLASH so that it doesn't get lost after reset but for runtime it has to be in RAM. To do this the linker script places is into FLASH but links it (resolves the symbolic references to this section) as if it was in RAM. To make this work, our boot-loader has to copy the data from FLASH to RAM before the main program runs.
• As for the uninitialised variables .bss, it's enough to link them for RAM. There is nothing to be placed in FLASH.

What the current boot-loader does

The assembly language file Crt0.S contains the boot-loader code and definition of the interrupt vector table.
The interrupt vector table basically only has to do one thing - set the reset interrupt entry to point to the start of the boot-loader code, so that it will be automatically called after the reset. For our purposes the boot-loader and the reset interrupt handler are therefore the same thing. The rest of the entries in the table are pointed to a dummy handler which contains only an infinite loop to prevent the micro from going crazy in case any of the other interrupts are asserted.
Then there is the code of the boot-loader itself. It does the three following things:

1. copies the initialised variables from FLASH to RAM. 
2. Zeroes out the .bss section (this is strictly speaking not necessary, but everybody else does it so I guess it's a good practice) 
3. Calls the main function.

The 3 boot-loader drafts

Now that we have explained how the current boot process works I will describe what me and Mr. Svoboda have came up with when discussing our approach to writing the new boot-loader. Basically, we came up with three drafts of how it could operate. The first one being the simplest to implement but the least useful and flexible and the last one being the most complex one:

1. The whole program will be loaded in FLASH but right after reset it will copy itself into RAM and run from there allowing it to rewrite itself in FLASH. After reset the new version of the program will run. This will be probably useful for experiments and testing only as there is no way that the whole program with all of its intended functions will fit just in RAM.
2. The program will be divided into two parts: lib and src. Lib will contain the boot-loader code and other code necessary for its function. Rest of the firmware will be placed in src. The idea is that lib will never be modified but it will be able to overwrite src. This would mean that lib would have to be 100% (I dare say maybe even 110%) finalised and debugged before we implement this solution.
3. The program will be placed in FLASH, as usual, but after receiving the command to enter boot-loader mode it will copy the necessary parts of itself into RAM and start executing from there, allowing it to overwrite itself in its entirety in FLASH and run the new version of itself after reset. 

Again, this is getting longer than expected so I will stop here for now and write about what I actually implemented and how tomorrow.

Working with the FLASH memory

As the title suggests, I have spent some time during the past week experimenting with the FLASH memory. On LPC13XX micros the memory is divided into eight sectors of 4 kB. Each of these is divided into eight pages of 256 bytes and these are in turn divided into eight 16 byte words.

When working with the FLASH memory there are some restrictions that have to be kept in mind:

1. The data have to be written into the memory in pages. That means that the size of the data has to be one page and the start address has to be a page boundary.
2. When erasing the memory it is possible only to erase whole sectors.
3. A page can be re-written at most sixteen times before it has to be erased (In combination with the previous restriction this means that the whole sector containing the page has to be erased).

Based on these restrictions, I and mr. Svoboda have drafted a specification for six new Elvis commands which are to be used for programming the FLASH memory.

The device's firmware and the tolpc program will be extended to implement these new commands.

The new FLASH programming commands

The FLASH programming works in the following way (in simplified terms): The micro contains a page-sized buffer and receives messages containing 16 bytes (one word) of data which are then written into the buffer. Once full, the whole buffer can then be written into the memory by sending another message containing the desired page address. The complete description of the programming messages can be found in the Elvis protocol specification, but I will give a brief rundown here as well:

1. Reset
Command to reset the micro. It does not really have anything to do with programming the FLASH memory, but it will be needed in the future to implement the boot-loader.

2. EraseCommands the micro to erase a range of sectors specified in the data part of the message.
3. LoadSends 16 bytes of data to be loaded into the buffer. Also contains an offset to specify where exactly to put the data into the buffer.
4. WriteContains a page address where the data from the buffer should be written.
5. Check-sumAfter receiving this command, the micro will compute a check-sum over all of the data contained in the page specified in the message. Then it will send in back to the master computer. The check-sum can be compared with the correct value to see if the data have been written into the memory without any errors. The CRC algorithm used is the same as for the Elvis messages. LPC13XX contains a hardware signature generator, which allows for the same functionality, but after some deliberation we have decided not to use it.

More work

Besides experimenting with the FLASH I have been working on creating a basic boot-loader. The way it is intended to work is that the program will be stored in FLASH, but right after reset it will copy itself into RAM and run from there; Allowing it to rewrite itself in the FLASH and run a new version of itself after reset. To accomplish this I have written a new linker script file and rewrote crt0.S, the file containing the code which runs right after reset. I thought I had them right but, unfortunately, it does not work, so it looks like I will probably have to take a look at some Cortex M3 debugging documentation and find out what is wrong.

I will let you know how that went next week.

Back home and another update

Hello again.
I have returned from my trip abroad and so now, I have the time to describe some things from my previous entry in more detail.

The communication protocol

The way in which the micro-controllers communicate with the master computer is specified by a proprietary protocol called Elvis. The structure of an Elvis frame is fairly standard and I don't think it's necessary to describe it here in detail; I mention it only because I may refer to the existence of this protocol later.

The tool-chain


Firstly, I must mention that there already exists a very good tutorial dealing with setting up a toolchain for Cortex-M3 micro-controllers and with introduction to their programming. We referred to it when setting up our own tools but we didn't follow it exactly. I will probably mention this tutorial several times in the following text.
The two essential tools are the C compiler and linker for Cortex-M3. We use a set of them made by one of our own faculty's employees Pavel Píša which are aviliable here.
Next thing to obtain is CMSIS - a set of C macros, definitions and functions which help with programming for Cortex-M3 processors by providing a simple access to various registers, functions to initialise the processor etc. Unfortunately it covers only the core Cortex-M3 functions and does not include anything to help with the peripherals and additional features present on LPC13XX boards.
In order not to have to figure this out from scratch, we used some code from examples in the LPC13XX MDK (microcontroller developement kit) made by a company called Keil and is freely available for download at their site (after registering). Again, these examples don't cover everything so there was still a lot of manual reading going on.

The firmware

In the previous section I have described tools that are freely available and that everyone can get.
Now I will write more about what I have been given to work with that is a part of the MagMan project.
I suppose that the following section will be of interest mainly to the people working on it.
Firstly, I received yet another piece of hardware - a little board that connects to an USB port on a personal computer on one side and to a serial communication PINs on the MagMan module board on the other side. Together with a piece of software called Tolpc, which I will describe later, it is used to program the micro-controllers .
As for software, I have been given the complete firmware up to this point which includes the following:
C files containing the program itself, linker files, Makefile, a piece of Assembly code containing the bootloader and the aready mentioned Tolpc utility.
The C files aren't really that interesting and I can't think of anything special that's worth mentioning about them. They are your standard C files.
The linker files are very similar to these described in the tutorial I have mentioned earlier. They tell the linker where to put what in the micros memory. Unfortunately I have not found a lot of tutorials or resources about linker files, save for the official documentation, but for now I didn't have to tinker with the files much.
The Makefile I got is very complex (I almost want to use the word byzantine here) and I can't honestly say that I understand all of it, but then again, it pretty much automates everything so I don't even have to. It also allows (together with several different linker scripts) to build programs and place them either in FLASH or RAM memory of the LPC13XX. Unlike the linker scripts there is a lot of resources out there dedicated to explaining makefiles around.
The single assembly file contains code of the initial boot-loader which is, again, very similar to the one described in the tutorial except, obviously, ours is written in assembly, not in C. After a reset the micro starts executing from the boot block in the FLASH memory, this is where the boot-loader is placed - It contains the interrupt vector table and interrupt service routine for the reset interrupt. The routine is called and it copies the rest of the program into RAM and then passes the execution to the main function.
The last thing to describe is the Tolpc utility. It serves, together with the hardware gizmo that I have already described, to program the micro. It utilises the LPC13XX ISP (in system programming) features which allow the micro to be (after reset) put into a state where it can be programmed over the serial interface.
What my goal is right now is to make pretty much the same thing as this, only working over the RS485 bus and using IAP (in application programming) functions instead of ISP.

Conclusion

I feel that this post is getting fairly long and I am getting less and less comprehensible, so I think I will stop here for now and leave the description of what I achieved in my experiments working with the FLASH memory for later.
Goodbye.

1st week progress report

I spent the past week at work mostly familiarising myself with the Lpc1313 and 1342 micro-controllers under the guidance of Mr. Petr Svoboda, who is the person responsible for all (or at least most) of the work on the firmware up to this point.
I will keep this entry brief and only summarise the results that were achieved, because I am leaving for a vacation tomorrow and I don't quite have the time to type out a long post. This also means that there will be no more posts for two weeks. I intend to go back to this one after I return and add all the necessary info and details.


Now for the results:
Firstly I set up the appropriate tool-chain for working with the Lpc1313 and 1342 micro-controllers. I keep referring to Lpc1342 even though it is not the micro-controller that is used in MagMan. This is because Mr. Svoboda 
provided me with a handy circuit board, which emulates the whole MagMan device and contains the Lpc1342 micro as well as a display, a push-button and some LEDs. These features make it more suitable for debugging and development, than the actual board which is really used. The differences between Lpc1313 and 1342 are minimal.
After that, Mr. Svoboda decided that the best way to familiarise myself with the development on these micros will be to give me "Homework" consisting of a series of tasks that my program should perform. I spent most of my time working on those and learning how to use the different peripherals on the Lpc1342, such as SysTick timer and watchdog timer; and I am now beginning to experiment with the process of rewriting the on-chip FLASH memory with data sent over the serial interface - a necessary step towards my goal of creating a boot-loader.

That's about it, see you in two weeks.

The intrudoctory post

Hello,

let's get the introduction out of the way first - My name is Viktor Kajml and I study robotics at the Faculty of electrical engineering at the Czech technical university in Prague. During this summer I have been chosen for a temporary job at AA4CC - one of the faculty's research groups. More specifically I will be participating in the development of a project called MagMan, which is a device for planar manipulation of objects by the means of generating and shaping magnetic field.

The purpose of this blog is document the progress of my work on this project and, provided there will be no unexpected complications, it should be updated about once every week.


A full description of the MagMan device can be found in this article but I will provide a brief summary here as well. Basically, it consists of an array of electromagnets, which are individually controlled to generate the desired magnetic field and move the test object (currently a metal ball) around. The array is composed of a number modular units, each of which contains four electromagnets, mechanical and electrical connectors that plug into the rest of the array and a microcontroller, which I shall describe in more detail, because it will be the focus of my work.

The used micro controllers are LPC1313 with the ARM Cortex-M3 microprocessor. Their function is to measure the current running through the four electromagnets, regulate it by generating the correct PWM signals and to communicate with the master computer via RS485 bus.

My task will be to improve and finalise the microcontroller firmware. This will involve completing several different sub tasks:

First of these will be to implement a bootloader that will allow re-programming of the micro controllers over the RS485 bus. This feature is sorely needed, as manual re-programming of each individual module over it's dedicated programming pins is very cumbersome and will only get worse as the array grows in size.

Next task will be to make the communication over the RS485 as fast as possible. Again this is necessary, if we want to extend the array, in order to keep the delays in communication with the master computer sufficiently short and to allow for an effective control.

Lastly, it would be desirable to implement a feedback controller to handle the regulation of the currents flowing through the electromagnets. As of right now the regulation consists simply of setting the desired duty cycle of the controlling PWM signal and no feedback is utilized. As a result the rise time of the value of the current is sub-optimal, which affects the overall control of the magnetic field.

I think that this is quite enough of work for right now and I hope that next week I will be able to post some results.