Wednesday, January 8, 2014

Developing STM32 microcontroller code on Linux (Part 3 of 8, building gdb)

The first post of this series covered the steps to build and run code for the STM32. The second post covered how to build a cross-compiler for the STM32. This post is going to cover how to build a debugger for the STM32.

Building a debugger isn't strictly necessary for developing on the STM32. However it can make certain debugging tasks easier, and it is relatively simple to do, so we'll do it here. As with the tools in the last post, the version of gdb used (7.6) worked for me. Your mileage may vary. If you fail to cross-compile gdb, then try a slightly newer or older version and try again on your development setup. If you can't build gdb, you can safely skip this step, though you may run into some problems later.

To build gdb, we'll assume you installed the tools to the path from the last post. If you changed path, you'll have to edit the PREFIX path below.

As before, we start out by exporting some environment variables:

$ export TOPDIR=~/cross-src
$ export TARGET=arm-none-eabi
$ export PREFIX=~/opt/cross
$ export BUILDPROCS=$( getconf _NPROCESSORS_ONLN )
$ export PATH=$PREFIX/bin:$PATH
The TOPDIR environment variable is the directory in which the sources are stored. The TARGET environment variable is the architecture that we want our compiler to emit code for. For ARM chips without an operating system (like the STM32), we want arm-none-eabi. The PREFIX environment variable is the location we want our cross-compile tools to end up in; feel free to change this to something more suitable. The BUILDPROCS environment variable is the number of processors that we can use; we will use all of them while building to substantially speed up the build process. Finally, we need to add the location of the cross-compile binaries to our PATH so that later building stages can find it.

Next we'll download, unpack, and build gdb:

$ cd $TOPDIR
$ wget ftp://ftp.gnu.org/gnu/gdb/gdb-7.6.tar.gz
$ tar -xvf gdb-7.6.tar.gz
$ mkdir build-gdb
$ cd build-gdb
$ ../gdb-7.6/configure --target=$TARGET --prefix=$PREFIX \
--enable-interwork
$ make -j$BUILDPROCS
$ make install
We download gdb, unpack it, then configure and build it. The flags to configure deserve some explanation. The --target flag tells gdb what target you want the tools to build for; that is, what kind of code will be emitted by the code. In our case, we want ARM with no operating system. The --prefix flag tells gdb that we want our debugger to be installed to $PREFIX. The --enable-interwork flag allows binutils to emit a combination of ARM and THUMB code; if you don't know what that is, don't worry about it for now. Assuming this step went fine on your development machine, there should be a binary in ~/opt/cross/bin (or whatever your top-level output directory is) called arm-none-eabi-gdb.

Tuesday, January 7, 2014

Developing STM32 microcontroller code on Linux (Part 2 of 8, building the cross-compiler)

The first post of this series covered the steps to build and run code for the STM32. This post is going to cover how to build a cross-compiler for the STM32.

The steps to build a cross-compiler are somewhat covered here and here. In theory, building a cross-compiler is a pretty straightforward process:
  1. Cross compile binutils, to get things like as (assembler), ld (linker), nm (list object symbols), etc.
  2. Cross compile gcc, which gives you a C and C++ compiler.
  3. Cross compile newlib, which gives you a minimal libc-like environment to program in.
However, there is a big gotcha. Not all combinations of binutils, gcc, and newlib work together. Worse, not all combinations of them build on all development environments, which can make this something of a frustrating experience. For instance, it is known that binutils < 2.24 does not build on machines with texinfo 5.x or later. Thus, on modern machines (like Fedora 19), you must use binutils 2.24 or later. Also, I found that the latest newlib of this writing (2.1.0) does not build on Fedora 19. Your mileage may vary, and this will almost certainly change in the future; the best advice I can give is to start with the latest versions of the packages and then slowly back off the ones that fail until you get a relatively recent combination that works. For the purposes of this post, I ended up using binutils 2.24, gcc 4.8.2, and newlib 2.0.0. This combination builds just fine on Fedora 19.

Now onto the steps needed to build the cross compiling environment. We first need to make sure certain tools are installed. We'll install the development tools through yum:

$ sudo yum install gcc make tar wget bzip2 gmp-devel \
mpfr-devel libmpc-devel gcc-c++ texinfo ncurses-devel
Next we fetch the relevant versions of the packages:

$ mkdir ~/cross-src
$ cd ~/cross-src
$ wget ftp://ftp.gnu.org/gnu/binutils/binutils-2.24.tar.gz
$ wget ftp://ftp.gnu.org/gnu/gcc/gcc-4.8.2/gcc-4.8.2.tar.bz2
$ wget ftp://sources.redhat.com/pub/newlib/newlib-2.0.0.tar.gz
Next we set some environment variables. This isn't strictly necessary, but will help us reduce errors in the following steps:

$ export TOPDIR=~/cross-src
$ export TARGET=arm-none-eabi
$ export PREFIX=~/opt/cross
$ export BUILDPROCS=$( getconf _NPROCESSORS_ONLN )
$ export PATH=$PREFIX/bin:$PATH
The TOPDIR environment variable is the directory in which the sources are stored. The TARGET environment variable is the architecture that we want our compiler to emit code for. For ARM chips without an operating system (like the STM32), we want arm-none-eabi. The PREFIX environment variable is the location we want our cross-compile tools to end up in; feel free to change this to something more suitable. The BUILDPROCS environment variable is the number of processors that we can use; we will use all of them while building to substantially speed up the build process. Finally, we need to add the location of the cross-compile binaries to our PATH so that later building stages can find it.

Now we can start building. We first need to build binutils:

$ cd $TOPDIR
$ tar -xvf binutils-2.24.tar.gz
$ mkdir build-binutils
$ cd build-binutils
$ ../binutils-2.24/configure --target=$TARGET --prefix=$PREFIX \
--enable-interwork --disable-nls
$ make -j$BUILDPROCS
$ make install
Basically we are unpacking binutils, doing an out-of-tree build (recommended), and then installing it. The flags to configure deserve some explanation. The --target flag tells binutils what target you want the tools to build for; that is, what kind of code will be emitted by the code. In our case, we want ARM with no operating system. The --prefix flag tells binutils that we want our tools to be installed to $PREFIX. The --enable-interwork flag allows binutils to emit a combination of ARM and THUMB code; if you don't know what that is, don't worry about it for now. Finally, the --disable-nls flag tells binutils not to build translation files, which speeds up the build. Assuming this step went fine on your development machine, there should be a set of tools in ~/opt/cross/bin (or whatever your top-level output directory is) called arm-none-eabi-*. If this didn't work, then you might want to try a newer or older version of binutils; you can't proceed any further without this working.

With binutils built, we can now move on to gcc:

$ cd $TOPDIR
$ tar -xvf newlib-2.0.0.tar.gz
$ tar -xvf gcc-4.8.2.tar.bz2
$ mkdir build-gcc
$ cd build-gcc
$ ../gcc-4.8.2/configure --target=$TARGET --prefix=$PREFIX \
--enable-interwork --disable-nls --enable-languages="c,c++" \
--without-headers --with-newlib \
--with-headers=$TOPDIR/newlib-2.0.0/newlib/libc/include
$ make -j$BUILDPROCS all-gcc
$ make install-gcc
Here we are unpacking gcc and newlib (which is required for building gcc), doing an out-of-tree build of the initial part of gcc, and then installing it. The flags to configure deserve some explanation. The --target flag tells gcc what target you want the tools to emit code for. The --prefix flag tells gcc that we want our tools to be installed to $PREFIX. The --enable-interwork flag allows gcc to emit a combination of ARM and THUMB code. The --disable-nls flag tells gcc not to build translation files, which speeds up the build. The --enable-languages flag tells gcc which compilers we want it to build; in our case, both the C and C++ compilers. The --without-headers --with-newlib and --with-headers flags tells gcc that it not to use internal headers, but rather to use newlib and the headers from newlib. Assuming this step finished successfully, there should be a file called ~/opt/cross/bin/arm-none-eabi-gcc, which is the initial compiler. Again, if it didn't work, then you might want to try a newer or older version of gcc; you can't proceed any further without this.

With the initial compiler built, we can now build newlib:

$ cd $TOPDIR
$ mkdir build-newlib
$ cd build-newlib
$ ../newlib-2.0.0/configure --target=$TARGET --prefix=$PREFIX \
--enable-interwork
$ make -j$BUILDPROCS
$ make install
Since we've already unpacked newlib, we skip that step. Here we are doing an out-of-tree build of newlib, using the compiler that we built in the last step. The configure flags have the same meaning as previously.

With newlib built, we can now go back and finish the build of gcc (the last step!):

$ cd $TOPDIR/build-gcc
$ make -j$BUILDPROCS
$ make install
This finishes the gcc build, and installs it to $PREFIX. That's it! You should now have a $PREFIX directory full of tools and headers useful for building code to run on the STM32.

Update Jan 8, 2014: Updated the formatting so it is more readable.

Monday, January 6, 2014

Developing STM32 microcontroller code on Linux (Part 1 of 8, introduction)

Recently I've been playing with the STM32, which is a small microcontroller made by ST. These seem to be pretty great microcontroller chips; they are relatively fast (depending on what model you get), have a decent amount of flash (up to 1MB), and have a decent amount of memory (up to 192KB). It is also easy to get development boards for them; there is a line of boards called the STM32DISCOVERY boards that are really cheap and easy to get. It is possible to work on these chips entirely with open source tools, which is important to me.

This series of posts will go through all of the steps necessary to develop on these boards. Note that all of this is covered elsewhere on the web, but a lot of the information is either outdated or scattered. I'll build all of the pieces from the ground up to get a working set of tools and binaries that you can use to develop your own STM32 applications.

To start with, I'm going to describe my hardware setup. I have a laptop running Fedora 19 x86_64. This is my main development machine, and this is going to be the host for everything I do with the STM32. For an STM32 board, I have an STM32F3DISCOVERY board, as shown here. However, note that for most of the posts, the exact board that you have isn't that important. As long as it is one of the STM32F*DISCOVERY boards, the steps below will mostly apply. The differences will become more important when we start to actually write code that deals with the GPIOs (as the GPIOs differ per board), but for the development environment they are really all quite similar.

This series of posts will do the steps in the following order:
  1. In order to do anything, we need a cross compiler. This is a set of tools that runs on our development environment (Fedora 19 x86_64), but emit instructions for our target hardware (STM32 ARM). Besides the C/C++ compiler, this also includes things like the assembler and linker. Part of the cross-compile toolchain also includes a minimal libc-like environment to program in, which gives you access to <stdio.h> and other familiar header files and functions. We will build a cross compile environment from binutils, gcc, and newlib.
  2. Once we have a cross-compiler, we need some way to debug the programs we write. The simplest thing to do here is to build gdb, the GNU debugger. Unfortunately we can't just use the system gdb, as that generally only understands how to debug and disassemble code on your development machine architecture. So we'll build our own version of gdb that understands ARM.
  3. With the debugger finished, we need some way to take the compiled version of our code and put it onto the target device. The STM32 devices use something called STLinkV2, which is a multi-purpose communication protocol (generally over USB). In order to upload our code to the device, we need a piece of software the speaks this protocol. Luckily there is OpenOCD, the Swiss Army Knife of communication protocols. We'll need to build a version of this that runs on our development machine, but knows how to speak STLinkV2. In this step we'll also build a configuration file that can communicate over STLinkV2.
  4. With the communications taken care of, we need a device library. This is basically an abstraction layer that allows us to talk directly to the hardware on the target device. For the purposes of these posts we are going to use libopencm3. This step will build libopencm3 for the target device.
  5. Once we have libopencm3 built, we have to know how to link programs so that they run on the STM32. This step will discuss linker scripts and command-line directives necessary to build programs that run on the STM32.
  6. Here we build our first simple program, upload it to the STM32, and watch it run! Finally!
  7. For bonus, I discuss running a simple Real-Time Operating System on the STM32, FreeRTOS. Using this will allow you to define several tasks and have the RTOS switch between them, much like tasks on a full-fledged OS. This opens up new possibilities and new problems, some of which will be discussed.
Whew, that's a lot of steps just to get the equivalent of "Hello World" running on the board. However, it should be educational and collect a lot of this information together in one place.

Friday, January 3, 2014

Oz 0.12.0 release

I'm pleased to announce release 0.12.0 of Oz. Oz is a program for doing automated installation of guest operating systems with limited input from the user. Release 0.12.0 is a bugfix and feature release for Oz. Some of the highlights between Oz 0.11.0 and 0.12.0 are:
  • Fixes to concurrent oz-install invocations
  • Python 3 compatibility in the test suites
  • Support for Ubuntu 12.04.3
  • Support for Mageia
  • Allow a MAC address to be passed in (instead of auto-generated)
  • Support for RHEL5.10
  • Support for Ubuntu 13.10
  • Use lxml instead of libxml2 for XML document processing (it has much better error messages)
  • Remove the unused "tunnels" functionality
  • Support FreeBSD 10.0
  • Remove deprecated functions from the Guest class
  • Speed up guest customization on guests that support NetworkManager
  • Follow subprocess commands as they are executed (makes debugging easier)
  • Ensure that any paths from the user are absolute, otherwise things don't work properly
  • Add support for OpenSUSE 13.1
  • Add support for Fedora 20
  • Add support for RHEL-7
A tarball and zipfile of this release is available on the Github releases page: https://github.com/clalancette/oz/releases. Packages for Fedora-19, Fedora-20, and EPEL-6 have been built in Koji and will eventually make their way to stable. Instructions on how to get and use Oz are available at http://github.com/clalancette/oz/wiki. If you have questions or comments about Oz, please feel free to contact me at clalancette at gmail.com, or open up an issue on the github page: http://github.com/clalancette/oz/issues. Thanks to everyone who contributed to this release through bug reports, patches, and suggestions for improvement.

Sunday, December 15, 2013

Release of ruby-libvirt 0.5.1

I'm pleased to announce the release of ruby-libvirt 0.5.1. ruby-libvirt is a ruby wrapper around the libvirt API. The changelog between 0.5.0 and 0.5.1 is:
  • Fixes to compile against older libvirt
  • Fixes to compile against ruby 1.8
Version 0.5.1 is available from http://libvirt.org/ruby: Tarball: http://libvirt.org/ruby/download/ruby-libvirt-0.5.1.tgz Gem: http://libvirt.org/ruby/download/ruby-libvirt-0.5.1.gem It is also available from rubygems.org; to get the latest version, run: $ gem install ruby-libvirt As usual, if you run into questions, problems, or bugs, please feel free to mail me (clalancette@gmail.com) and/or the libvirt mailing list. Thanks to everyone who contributed patches and submitted bugs.

Monday, December 9, 2013

Release of ruby-libvirt 0.5.0

I'm pleased to announce the release of ruby-libvirt 0.5.0. ruby-libvirt is a ruby wrapper around the libvirt API. Version 0.5.0 brings new APIs, more documentation, and bugfixes:
  • Updated Network class, implementing almost all libvirt APIs
  • Updated Domain class, implementing almost all libvirt APIs
  • Updated Connection class, implementing almost all libvirt APIs
  • Updated DomainSnapshot class, implementing almost all libvirt APIs
  • Updated NodeDevice class, implementing almost all libvirt APIs
  • Updated Storage class, implementing almost all libvirt APIs
  • Add constants for almost all libvirt defines
  • Improved performance in the library by using alloca
Version 0.5.0 is available from http://libvirt.org/ruby: Tarball: http://libvirt.org/ruby/download/ruby-libvirt-0.5.0.tgz Gem: http://libvirt.org/ruby/download/ruby-libvirt-0.5.0.gem It is also available from rubygems.org; to get the latest version, run: $ gem install ruby-libvirt As usual, if you run into questions, problems, or bugs, please feel free to mail me (clalancette@gmail.com) and/or the libvirt mailing list. Thanks to everyone who contributed patches and submitted bugs.

Saturday, November 9, 2013

Writing Ruby Extensions in C - Part 13, Wrapping C data structures

This is the thirteenth in my series of posts about writing ruby extensions in C. The first post talked about the basic structure of a project, including how to set up building. The second post talked about generating documentation. The third post talked about initializing the module and setting up classes. The fourth post talked about types and return values. The fifth post focused on creating and handling exceptions. The sixth post talked about ruby catch and throw blocks. The seventh post talk about dealing with numbers. The eighth post talked about strings. The ninth post focused on arrays. The tenth post looked at hashes. The eleventh post explored blocks and callbacks. The twelfth post looked at allocating and freeing memory. This post will focus on wrapping C data structures in ruby objects.

Wrapping C data structures

When developing a ruby extension in C, it may be necessary to save an allocated C structure inside a Ruby object. For instance, in the ruby-libvirt bindings, a virConnectPtr (which points to a libvirt connection object) is saved inside of Libvirt::Connect ruby object, and that pointer is fetched from the object any time an instance method is called. Note that the pointer to the C structure is stored inside the Ruby object in a way that the ruby code can't get to; only C extensions will have access to this pointer. There are only 3 APIs that are used to manipulate these pointers:
  • Data_Wrap_Struct(VALUE klass, void (*mark)(), void (*free)(), void *ptr) - Wrap the C data structure in ptr into a class of type klass. The free argument is a function pointer to a function that will be called when the object is being garbage collected. If the C structure references other ruby objects, then the mark function pointer must also be provided and must properly mark the other objects with rb_gc_mark(). This function returns a VALUE which is an object of type klass.
  • Data_Make_Struct(VALUE klass, c-type, void (*mark)(), void (*free)(), c-type *ptr) - Similar to Data_Wrap_Struct(), but first allocates and then wraps the C structure in an object. The klass, mark, free, and ptr arguments have the same meaning as Data_Wrap_Struct(). The c-type argument is the actual name of the type that needs to be allocated (sizeof(type) will be used to allocate).
  • Data_Get_Struct(VALUE obj, c-type, c-type *ptr) - Get the C data structure of c-type out of the object obj, and put the result in ptr. Note that this pointer assignment works because this is a macro.

An example will demonstrate the use of these functions:

 1) static VALUE m_example;
 2) static VALUE c_conn;
 3)
 4) struct mystruct {
 5)     int a;
 6)     int b;
 7) };
 8)
 9) static void mystruct_free(void *s)
10) {
11)    xfree(s);
12) }
13)
14) static VALUE example_open(VALUE m)
15) {
16)     struct mystruct *conn;
17)     conn = ALLOC(struct mystruct);
18)     conn->a = 25;
19)     conn->b = 99;
20)     return Data_Wrap_Struct(c_conn, NULL, mystruct_free, conn);
21) }
22)
23) static VALUE conn_get_a(VALUE c)
24) {
25)     struct mystruct *conn;
26)     Data_Get_Struct(c, struct mystruct, conn);
27)     return INT2NUM(conn->a);
28) }
29)
30) void Init_example(void)
31) {
32)     m_example = rb_define_module("Example");
33)     rb_define_module_function(m_example, "open", example_open, 0);
34)     c_conn = rb_define_class_under(m_example, "Conn", rb_cObject);
35)     rb_define_method(c_conn, "get_a", conn_get_a, 0);
36) }

On lines 32 and 33, we define the Example module and give it a module function called "open". Lines 34 and 35 define a Conn class under the Example module, and gives the Conn class a "get_a" method. Lines 14 through 21 are where we implement the Example::open function. There, we allocate memory for our C structure, then use Data_Wrap_Struct() to wrap that C structure in a ruby object of type Example::Conn. Note that we also pass mystruct_free() as the free callback; when the object gets reaped by the garbage collector, this function on lines 9 through 12 will be called to free up any memory. Now when the user calls "get_a" on the Example::Conn ruby object, the function on lines 23 through 27 will be called. There we use Data_Get_Struct() to fetch the structure back out of the object, and then return a ruby number for the integer stored inside. Update: added links to all of the previous articles. Update Jan 27, 2014: Updated the example to fix the use of ALLOC(). Thanks to Thomas Thomassen in the comments.