I’ve been asked many times why bitbake is useful, what it’s good for, and why anyone should bother to learn how to use it. From the down-in-the-weeds perspective I see bitbake fitting into a continuum of tools that developers already use. Compilers (like gcc) are used at the file-level to turn source code into binaries. At the next level there are make and configure which tie together many source files into a portable source package. And I see bitbake at the next level up, fetching packages from the internet or a SCM, applying patches, setting configure, make, and gcc variables, and selecting which binaries are pulled from the package, as well as defining how the binaries are packaged for delivery. Bitbake provides control down into software packages, and is extensible to allow additional steps like static analysis or testing.

While bitbake is capable of managing downward into the details of a particular software package, it is also aware of other packages - make and gcc don’t manage inter-package dependencies. Package dependencies are nothing new, and tools like RPM try to handle this also. Bitbake does allow you to differentiate between build dependencies and run-time dependencies though, so your system image won’t be polluted with extra packages that are simply needed on the host to build the software.

Finally, bitbake is capable of grouping related packages together into tasks. Tasks are not packages per-se, even though they are defined as bitbake recipes. An example would be a “printer” task that depends on cups, foomatic-filters-ppds, and ghostscript-gpl. By grouping these packages together into a task, you can now set build options for that particular task all the way down into the gcc and configure level. This task can also be re-used across multiple projects if your product line needs it.

In summary, bitbake helps you build your custom Linux distribution: it is compatible with thousands of recipes from various OpenEmbedded-based distributions, it provides low-level granular control over the build and install of each package, and provides higher level package groupings so you can leverage larger functional units. Done properly, a build can simply be bitbake my-product-image.

On Tue I gave a talk at RTECC Santa Clara about a source-based approach to embedded Linux. The gist of the talk was based on a few points:

• Standardized distributions don’t provide complete solutions for embedded developers
• Embedded developers heavily customize - add/remove/modify many elements in the software stack
• Every device is unique, every software stack is a Custom Linux Distribution
• A codified (standardized, programmatic) method for defining a CLD improves efficiency
• Bitbake (the tool, not OE the distro) is a step in the right direction for such a standard

The audience was mostly senior engineers, and I saw a lot of heads nodding as I introduced the first couple of points. It’s been my experience that senior engineers usually end up building and maintaining the glue-ware infrastructure for putting together a CLD. When I got into the bitbake portion of the talk I had to clarify that I was talking about bitbake-the-tool, not bitbake-plus-openembedded-distribution. This was clarified when I showed how we do this in MVL6.

Overall there was a warm response to the notion of providing a standardized way of creating a CLD, but even separating bitbake from the OE distro requires some work. Open source hasn’t quite solved the problem yet, which is why MontaVista is trying to push bitbake as a process standard, regardless of the underlying distribution.

In the last week, my employer, MontaVista Software, was acquired by Cavium Networks. Hot on the heels of the Intel/WindRiver acquisition, people are now asking us why Cavium’s competitors in the semiconductor business would continue to work with MontaVista. Aside from our public committment to operating independently from Cavium, I think MontaVista has a unique value that will benefit semiconductors and all of embedded Linux.

If we could capture the value of open source software in a word, I would call in Innovation. But innovation in software often doesn’t work well within process. Not to mention, open source is still very Wild West in behavior. Semiconductor manufacturers understand and apprciate process, because silicon is expensive to produce and bad process leads to lost money very quickly.

In MVL6, MontaVista is building out the software assembly process for the embedded Linux industry, aligning the embedded Linux supply chain. Every embedded device has a custom software stack - its own unique Linux distribution. So the need in the industry is to enable configuration, assembly, and maintenance of custom distros. The MontaVista Integration Platform addresses the configuration, assembly, and maintenance processes, and we will continue to build out functionality that makes this easier for our customers.

But beyond the MontaVista customer base, we are working with all of our semiconductor partners on defining and building out the procuess around building custom Linux distributions. I think the entire industry will benefit from these activities, and its in the best interest of all semis to participate and provide their input, and so far these process-oriented manufacurers seem to understand this value and are willing to participate.

A recent rant led me to read Matt Porter’s slides titled “Mythbusters: Android“. Matt’s analysis is a laundry list of Linux no-no’s found in Android. I became curious about whether any of these technical issues would really matter to Google, so I stopped by the OHA site to look at what they claim about Android, and do the analysis from that direction.

The OHA is pushing the notion that it’s all about the applications (copying Apple’s “there’s an app for that“):

• “The platform will continue to evolve as the developer community works together to build innovative mobile applications.”
• “Android breaks down the barriers to building new and innovative applications.”
• “Android provides access to a wide range of useful libraries and tools that can be used to build rich applications.”

This notion that it’s all about the apps seems to be working well for Apple. One big difference between Apple and OHA is that Apple only needs to support its own platform, and Apple owns the entire software stack. This lets Apple provide developers with well-defined boundaries around what the platform can do. Android, however, isn’t playing in such a simple environment, and needs to deliver benefits beyond a single platform if it wishes to live up to its claims.

Matt points out that there are several problems with the Android stack that could prevent or hamper development on Android, at least for traditional Linux/UNIX developers.

• Bionic: Android’s own libc implementation is for ARM/x86 only, has only partial pthreads support, has no IPC support, has no STL support. That’s a lot of stuff that Linux app developers need to work without.
• Device Management: Standard Linux distros use the highly-configurable udev and hal components to manage system device enumeration and hotplug support. Android has hard-coded devices into their own init system, so developers will need to modify the init system for any “non-standard” hardware.
• Power Management: Hard-coded for handsets. If you have a disk or other non-handset devices, you’re on your own.

So the benefit to Android device users may be the applications from the Android Market. From the perspective of device manufacturers, you get a high-investment, low-differentiation, ARM-based phone, that can participate in the Android market. Does Google care about the technical issues Matt points out? We’ll have to see whether Google invests in lowering the barriers-to-entry on Android, making system-level configuration easier, supporting a larger number of architectures, and providing standard programming interfaces that developers expect.

I read this blog post by an EEE PC developer who sounds frustrated about trying to keep his users up-and-running on Linux. It seems that every time he fixes a problem, the upstream developers create another problem somewhere else. While there are issues in many parts of the software stack, I see this kind of problem here at MontaVista most often in the kernel area.

Back when Linus & Co used odd version numbers for dev releases and even numbers for production releases (e.g. 2.5 & 2.6), users of Linux got some clear expectations from the kernel development team, and commercializers got some breathing room for packaging up their Linux products. Since the inception of the 2.6.x.y development methodology, the kernel guys have drastically increased the volume and rate of changes they pull into the kernel, and no longer provide any breathing room for users and distributors to stabilize.

Making this problem worse, there is also the widely-held belief that the latest kernel is the best kernel.  This just is not the case, as the kernel release trees are peppered with error-prone releases. I think this guy’s problem (and that of his users) comes with trying to keep up with the Community - which is about as useful as keeping up with the Jones’.  Unfortunately, everyone seems to think this is what they need to do.

How will Linux users be able to find a stable, usable system, that provides bugfix updates, and doesn’t ruin total-system functionality?

Google recently sent a cease and desist order to a developer who was including the proprietary Google apps in his Android ROM. The developer, cyanogenmod seems to have no hard feelings about the situation and is willing to work with Google. But the broader community appears to be dissatisfied with the situation. I see a couple of interesting angles on this, Google’s need to own their applications, and Google’s desire to own as much as possible.

First, Google has the right to own their applications, but beyond that they also need to provide assurance to the public that Google-branded applications are safe and secure. This is nothing new, the Symbian Signed program was designed to provide this type of assurance to device makers, and network operators. It seems necessary and essential that Google protect the distribution channel of their applications.

On the other hand, Google also has a desire to exert control over the market to direct people to their web properties. The Android platform was sold to developers as an attractive, new open-source project, the next big thing. But ultimately Google wants to make sure their apps show up on whatever devices hit the market - All Your Internets Belong to Us. It’s my guess that this is just the reason that Apple rejected the Google Voice app from the iPhone. Google’s application strategy could have the effect of turning the iPhone into just-another-Google-device.

For the time being, device makers and network operators seem to be warm to the Android platform since it gives them time-to-market advantage for developing interesting devices. But will the market tolerate a Google strategy that effectively renders all players subservient to Google? And does this recent C&D Order represent the first Darth Vader move for Google?

As a Linux marketer I try to keep track of open source alternatives. For example, I occasionally meet engineers who prefer to use BSD for various reasons. Just the other day I came across the Haiku project, which is an open-source project to revive BeOS. I’d never used BeOS, but I remember a co-worker from NCSA going to work there many years back, so I decided to give the alpha a try and see what I’d been missing.

I downloaded an ISO which I booted into VMWare, and after just a few seconds I was asked whether I wanted to Install, or proceed to the Desktop. I liked that. Once I got to the desktop, I’ll admit I was a bit lost. I started clicking on icons to see if I could find anything interesting. I quickly found the documentation, which appeared to be aimed at developers, not users. The docs launched in a browser, with a link to Access at the bottom - I didn’t investigate that relationship any further. I then launched the Wonder Brush drawing app which performed fairly well IMHO for inside a virtual machine.

For now Haiku appears to be a novelty. I had fun spending 15 minutes playing with it, but it’s not immediately apparent how it can become relevant beyond a few enthusiasts or hobbyists. And if you’re working on an open-source implementation of OS/2 Warp 4, let me know.

Last week I celebrated my 5th anniversary at MontaVista, and was telling a co-worker a story about some previous engineering experiences.

At a previous employer I was on a small engineering team responsible for producing a commercial Linux distribution. Our “build system” was 1) a guy on the East Coast and, 2) some scripts on his workstation - the workstation could never be updated or modified lest we risk breaking the build.

A large conference was coming up and we needed to demo our latest software, so we crunched to get the latest bits checked in and our build guy produced some DVD images and FedEx’d them to San Francisco. We receieved the DVDs on a Friday afternoon, only to discover that upon booting the new OS the kernel panic’d.

With less than 2 days to get the system ready to ship over to the conference, and the East Coast build guy off for two weeks of vacation, the two West Coast engineers (myself and a co-worker) spent the weekend in the office pulling apart SRPMs, manually patching sources, and doing whatever we needed to cobble together something that would boot and demonstrate the applications we had ready for the show.

The products we shipped at that company were enterprise-oriented, and so the build system was really only critical to our internal process - our customers never ripped the system apart, even though it was Linux-based and they had every right to do that. Now that I’m out of Enterprise and in Embedded, I see that customers almost always want to take our products apart and put them back together to customize for the embedded devices they’re making. IMHO, this makes a build system a basic necessity of any embedded Linux product, and I’m glad MontaVista has been able to rally around the open-source build engine
bitbake and ship it in our latest product.