Suppose you’re writing a C program and want to open a file and read some data. The obvious way to do this is to use the C stdio library, which isn’t terribly well designed, but is at least standard. And it will for the most part work the same on Linux, Unix, and even Windows. The basic abstraction is the stream, represented as a FILE *.
For simple programs will simple requirements, stdio works well. But it is not extensible in a standardized way. In other words, the “top side” of the stdio stream abstraction is defined, but the bottom side is not. There is no way to write a portable library that can hook into stdio and provide streams that access different underlying data types than a simple file in a filesystem provided by the OS.Suppose you want to do fancier stuff that stdio doesn’t support well. On Linux or Unix, you might use system calls like open() and read(). These are portable to all Posix-based (and Posix-like) systems, but not to Windows, which has its own set of system calls. With open(), you get a file descriptor (a small integer), which you pass to the other system calls as a handle. If you want to use stdio for some of your file access, there is a function fdopen() that takes a file descriptor and creates a stream. But don’t try to mix and match stdio and syscall I/O; siince stdio normally does buffering, you won’t like the results.
If you were writing a program using GTK+, you might use Glib’s g_io_channels. They are portable to Windows, and play nicely with the Glib and GTK+ main loop. However, you can’t use either the stdio or Posix syscalls on them. You’re stuck with the Glib API. Fortunately this API does provide a reasonable set of functions, and they are generally safer than the stdio functions. And unlike stdio, the underside of the g_io_channel is extensible.
On the other hand, you might like to have an API that allows you to read files that are on remote servers using various protocols, perhaps HTTP or FTP. If your operating system has the ability to “mount” volumes from those remote servers, you can probably use any API you like. Otherwise you need a user-space library like GnomeVFS. And (surprise!) it provides its own API. It might have been nice if it used g_io_channels, perhaps adding a few additional calls, but sadly it does not.
And on the gripping hand, you might want to access local files but which happen to have structured content (e.g., a ZIP file), and transparently access members of the structure as if they were first-class files. You might have thought that GnomeVFS would help you here, and perhaps for a local .tar.gz file it might, but for a local .zip file you need a different library, libgsf2. Naturally, libgsf2 provides its own APIs that are not particularly similar to any of the others.
When I first saw libgsf2 I was pretty excited about it, because I had been searching for a good library to access the contents of ZIP files for some time. There were a few other such libraries out there, but they all either had technical limitations, ugly APIs, or overly restrictive licenses. libgsf2 has a fairly decent API (though very poorly documented), and uses the LGPL license.
I considered hacking libgsf2 to provide either a more stdio-like interface, or a more g_io_channel like interface. I exchanged a bit of email with one of the libgsf2 maintainers, who thought the former was a good idea but didn’t like the latter, which I’d prefer. If libgsf2 could provide a g_io_channel interface, then a wrapper could be written around the g_io_channel topside API to provide a stdio stream like interface (but not actual stdio streams).
Then I started looking at GnomeVFS. In addtion to support of various remote access models, it adds asynchronous I/O to the mix. Though the GnomeVFS async I/O API is apparently somewhat controversial, and some people think it should be replaced. I don’t think I want to go down that path myself.
My current thinking is to write a g_io_channel wrapper for libgsf2, but not change the implementation of libgsf2 itself unless I find that I need to add something to be able to support the g_io_channel. And once that’s done, to write the stdio-like wrapper for the g_io_channel.
But I’d really like to see the C standards committee add to a future revision of the C standard some specifications for interfacing to the underside of stdio streams.