Caveat Lector

So, I established last time that unique identifiers and cite-able text are technological hurdles needing solution, if we’re to have ebooks that prove useful.

There’s a problem with unique identifiers, though. No, not the problem of the same identifier accidentally getting slapped on two books. These things happen, and can be worked around. I mean something different: how do you know that a chunk of bits with a given unique ID is a proper copy of the ebook that deserves to have that unique ID?

I mean, if you think nobody’ll spoof an ebook, for fun or in malice, you haven’t been hanging around either the Internet or academia very long.

I’ve heard about watermarking as a solution for this kind of thing, but it seems overelaborate to me. I don’t know much about checksums and hashes and whathaveyou, but the idea (insofar as I understand it) appeals: the ebook contains a short chunk of metadata representing the result of a calculation carried out on the ebook itself. Grab the ebook, do the calculation, check it against the metadata, scream loudly if there’s a discrepancy. Fairly simple to do, difficult or impossible for a spoof to defeat.

An additional advantage to this scenario is protection against errors in transmission. If the ’net hiccups and drops half the ebook by accident, the checksum won’t check out, and the reader will know there’s a problem.

Either way, or some third way I’m not smart enough to imagine, we need to figure out something.

Eh, I don’t think I’m going to answer the latest Rothman ranting. I agree with much of it, strongly STRONGLY disagree with the conspiracy-theory aspects (why attribute to conspiracy what is simpler to explain as incompetence?), and am still waiting (with breath I dare not bate lest I suffocate myself) to see anything practical come of it.

I’m more interested in the question of what technological standards ebooks really need, over and above what already exists or can be forecasted to exist shortly. I find that my thoughts boil down to the question, “How is an ebook different from a website?”

To narrow it even further, I am considering questions of text artisanry only—how ebooks need to be constructed and how ebook readers need to make those constructions work. I don’t intend to waste brainspace on DRM. I can make a perfectly happy life for myself figuring out how to best deal with the ebooks people want us to have; I’d rather not cater to paranoia and greed.

So here’s a starter: Ebooks, like their print counterparts, are persistent in a way that websites can, but need not, be.

This has quite a few implications. One: a given ebook “edition” needs a unique and persistent identifier. Which leaves us wondering what an “edition” is. If HappyBook and BouncyBook build ebooks from the same source, are they the same edition? The answer heretofore has been “no,” but I personally think that’s a dreadful answer. They’re no more different than the VHS and DVD versions of the same movie—possibly less, given all the extras that DVDs carry these days. And in my local library’s OPAC, that’s “format” information, no more.

So what separates one electronic edition of a book from another, if not format? Well, all right, what separates one paper edition from another? My answer (which may not square with anyone else’s): publisher, changed content, and page numbers. Why page numbers? For citation purposes, of course.

Ah. But ebooks don’t have page numbers. The first generation of OEB-based readers allowed font size change on the fly, which was terrific for folks needing large print, but hell on anyone trying to cite a section.

So that’s another sine qua non, to my mind. We have to solve the citation problem. And then pagination becomes irrelevant, format (defined as “end-user format,” and before you think I’m advocating a repeat of the Format Wars, consider that Braille and speak-aloud versions can come from the same XML source that becomes the kind of ebook we sighted folk are more familiar with) becomes irrelevant. All that matters, one edition to another, is whether the content or the publisher has changed, which is how things ought to be.

The only way the identifier and citation problems get solved is with standards. I think existing or in-progress standards can tackle identifiers, myself—but the citation problem is (to my mind) technological. Every format for a given edition of an ebook has to be able to spit out the same citation for a given spot in that ebook, and by the same token has to find the same spot in that ebook given the same citation.

XPointer? Maybe, maybe… but something, and it’ll take a standards body to work out the something.

What else? Stay tuned. I have some more ideas on this score.

The question becomes, then, whether the OEBF is a suitable venue for continuing development of electronic-book specifications.

Frankly? No.

Shortly after I started work at OverDrive, I heard Steve speak at an OEBF plenary. His attitude toward the OEBPS was that the OEBF had already done the standards thing, wrung what there was to wring from it—and it was time to Move On to more fun and exciting things. Public relations, mostly. And trade-org stuff, whatever it is that trade orgs do (I’ve never been sure).

Hardly coincidence that PubStruct withered inside the next year. And from respect for the principals involved (some of them, at least), that’s all I’m going to say about that.

I’m dead sure the OEBF can’t get back the people it lost over that year. That isn’t entirely because of bad memories, either; part of it is that major contributors to the first effort have had their employers back off on the whole ebook concept, and thus can’t get leeway to participate any longer. (I am thinking particularly of Microsoft and Xerox PAL here; the demise of Gemstar matters quite a bit also.) The original technical talent pool has been quite thoroughly muddied.

Nor do I think the OEBF can put together another one. Ebooks are a hard sell to technical talent these days. About the only other thing that would attract more technical talent is existing technical talent, which they just don’t have any longer. So even if they realize their mistake, I seriously doubt they can correct it at this late date.

Myself, I think the OEBF is moribund, not worth the energy that Mr. Rothman is expending on it. It’s easy to be angry at them for some of the chances they threw away. I used to be pretty ticked off myself. Better, however, we should focus on the future—which is brighter than one might think.

I see three bright spots in e-text today. With any luck at all, they’ll converge to finish the job that PubStruct started. On the technical side, there’s James Clark and Murata Makoto, who are tackling the nasty namespace-combination issues that PubStruct was trying (rather fruitlessly) to find a metadata solution to. There is also Relax NG, which solves the grotesque problems PubStruct was having with DTDs. (Well, most of them. There’s still the warty old HTML entity set. Sigh.)

On the book side, there’s the accessibility people, and the nice state legislators who are starting to require that textbook publishers use e-text and electronic production methods to make their wares accessible for the visually-impaired. I don’t have the whole story here—but it sure does bear watching. If we can get textbooks, which are some of the hardest book-production jobs known to man, made into markup, we can bloody well do anything.

And on the social side—the demand side—there’s open-access scholarship. I remember my dad the professor’s article file—I helped fill it when I was a youngster. Three big unwieldy file cabinets, organized by author last name and date—and since most papers had multiple authors, Dad couldn’t ever find an article he wanted anyway. Give academia a wireless gadget with a good display that finds this stuff, downloads it, and organizes it for fast retrieval and perusal, and they’ll stampede a path to the ebook door. I mean, really, it just isn’t that hard to beat the current state of e-reserves (which are flourishing mightily).

I’m actually of the opinion that former ebookers who can’t climb on one of the above-mentioned bandwagons (and if they can, they should, by all means) should sit back and wait. Wait. Wait and see. Interesting times coming.

So what’s Rothman on about?

Well, one thing is a follow-the-money hit on OEBF management in general and Steve Potash in particular. I don’t particularly like either the OEBF as currently constituted or Steve (and I’ve made no particular secret of it, either), but Rothman’s hit is absurd and his proposed solution drastically incomplete, and it doesn’t take any particular knowledge of the OEBF or of Steve to see why.

Yes, the OEBF is run on fees from its members. Duh. That’s never been a secret. Yes, they’re a trade group, made up of and answerable to their industry-drawn members rather than the general public. That’s been so all along, too. Yes, their management is drawn from their members’ management. Where the hell else are they supposed to get it, hm?

See, here’s the thing. That’s how just about all standards bodies work. It’s not specific to the OEBF; the W3C works the same damn way. The IETF doesn’t, admittedly, but the IETF also has a real problem with being ineffectual because of shifting membership and no direction. I have asked, over and over and over again, that people read Liora Alschuler’s brilliant article about this. GO READ IT NOW, PLEASE; here’s a quote most apropos to Rothman’s criticisms:

There is a purism about standards writing that seems to say, if you are accepting money, somehow your motives are tainted, your work is suspect. Yet very few people do this work without financial support. “Paid” and “volunteer” merely designate whether the funding is through an employer (who sees a business interest in the outcome) or through a public agency (which sees a public benefit in the outcome.)

Yes, it’s a problem—standards bodies are only answerable to the public good insofar as their individual members are idealists (and we’re damned lucky that a fair few smart techies seem to have an idealist streak) and can get away from their sponsors’ wishes long enough to be idealists. I don’t have a pat answer to that, but it certainly informs my wish that librarians would get more involved with standards development.

Rothman doesn’t have an answer either, from anything I’ve seen. It’s all very well to call for a new organization, but how will it be funded? How will it be led? Who will its technical experts be? How will they be supported in their work? (And I don’t mean “who pays them?” though that is a fine question; I mean “who does their time-consuming logistical and editorial gruntwork?” The W3C pays people to get gruntwork done. PubStruct lucked into me, and couldn’t replace me when I left.) What stops the new org from being just another OEBF? Who gets industry buy-in to whatever the new org produces? Whence comes the new org’s credibility?

It’s just not as simple as calling Steve Potash the bad guy. For what it’s worth, my impression is that Steve believes what he says. He genuinely believes that DRM is both necessary and inevitable. He believes that without the mass-market, ebooks will wither and die. And he really wants ebooks. He really does. Yes, he aligns both OverDrive and the OEBF around his beliefs. How could he reasonably do otherwise? I think he’s wrong in a lot of ways, mind you—but he’s honestly wrong, not venally wrong.

Nor is Steve on the take from the OEBF. Candidly, I suspect his presidency has cost him and OverDrive far more than it’s ever made up in business gained. And there is no—I mean, NO—chance of Steve doing anything dishonest with OEBF funds. (Dumb, yes; I see the openanebook.org page now points back to the OEBF, after the silly PR stunt they tried. But dishonest, no.) For one thing, there’s oversight. For another, Steve is a lawyer by trade and bloody well knows better. For a third—and you’ll have to trust me on this one—that’s very, very not Steve.

Sure, the contacts he makes via OEBF work are important to him and his business. That’s a key reason businesses join standards bodies. And it’s a key reason for individual people, for that matter—I still have plenty of people in my card-file I met through OEBF, and they’re still important to me professionally. If that’s an inherent conflict of interest, then I’m as guilty as Steve.

And if it’s not, then Steve isn’t guilty either.

I’d really appreciate it if Mr. Rothman would quit smearing Steve Potash. I’d appreciate a public apology, too, and a private one would be eminently appropriate in addition. I think attacking Steve was a cheap and ultimately pointless attention-grabbing ploy. There are indeed real issues, technical and social, and my next post(s?) will try to address them and suggest a way forward, but whatever his mistakes and possibly mistaken beliefs, Steve Potash absolutely doesn’t deserve to have his integrity called into question, not without vastly better evidence than Mr. Rothman has presented. Or, indeed, vastly better evidence than I think actually exists.

Well, this is one of the awkwarder positions I’ve found myself in. And I suspect it’s one that won’t lend itself to the usual length of a single CavLec blog entry. So be it, I suppose… I can always write more than one.

This post had better carry all the disclaimers and all the history. Off I go, then.

David Rothman has called twice recently for an end to the Open eBook Forum. I’ve got my own opinions on that, and me being me, you’re all going to be subjected to them. But first…

I started working with the OEBF on the OEB Publication Structure in May 2000, when the OEBF formally incorporated. I was at the time working for Impressions Book and Journal Services, which became an OEBF member and paid for my time and travel expenses as I did OEBF work. I was in short order chosen scribe to the Publication Structure working group, and in that capacity I edited the 1.0.1 release of the Publication Structure.

I left Impressions in May 2001 intending to make a go of TAG, but Steve Potash of OverDrive hired me instead. I worked for OverDrive until March 2002. Steve came within a millimeter of firing me once, but in the end I walked under my own power. TAG had to wait, though, owing to a non-compete agreement signed with OverDrive which I honored to the letter.

I could not, however, make time to continue with the OEBF after leaving OverDrive—and certainly I didn’t have travel money. Truth be told, too, I wasn’t happy with where I saw the organization going, or with what looked like (and in hindsight was) a general disengagement on PubStruct. We just weren’t accomplishing anything. I resigned as PubStruct scribe.

Take all this into consideration while you read. Myself, I have not yet read any reactions to Rothman’s articles (though they got a Slashdot mention yesterday), nor have I spoken with anyone associated (now or since) with the OEBF, OverDrive, or Impressions about them. (Jon Noring alerted me to the articles’ existence, but only with a bare notification—he said nothing other than that, nor did I.)

Whatever I say, it’s just me saying it. I don’t speak for any person or organization but myself and TAG, and any agendas I have I’m doing my best not to hide. I may, in fact, get in trouble for opening my mouth on this—but so be it; openness is more important.

So that’s where I’m writing from.

A basically sensible discussion of how ebooks went off the rails ends thusly:

It is now clear that the development of the appropriate digital reading technologies and interface, as well as a properly formatted digital repository of millions of books, will take five to ten years to develop.

’Scuse me, how long?

He’s wrong in two directions. Appropriate reading technologies and interface already exist, for a fairly minimal level of “appropriate.” What are you doing right this minute? Reading onscreen.

Technologies and interfaces that match or exceed print? Gonna take a hell of a lot longer than five to ten years, buddy, especially now that “everyone knows” that ebooks don’t work. Nobody’s doing research that I’m aware. Nobody’s making more prototypes. I’m about as close as it gets to a “deep thinker” when it comes to ebook progress. All the things we need to do to realize the potential of electronic text are just not being done.

I mean, sheesh, most incunabula are grotesque—ever seen one? It wasn’t years, it was centuries before we got anything to equal an illuminated manuscript. That’s how this transition works, too.

There’s a heartbreaking lot to do. Some of it is pure mechanical engineering, and I have no idea how those people work. But some of it is recreating some wheels. Navigation has got to improve, radically. We also need to sort out a structure for citing markup, something as convenient as the page. (I think the eventual solution is single-click access to an XPath, myself—but boiled ebooks don’t currently make available enough structural information for this to work.) We have to fix the indexing problem, and the random-access problem. None of this is easy.

That said, there are bright spots. Marked-up text is accumulating, not in floods, but drips and dribbles do add up after a while. Some of the markup is downright crap, admittedly—but it’s still marked-up text, and crap can be de-crapified given patient peasants. And the visually-disabled—they’re rocking the world. They’ve got stuff in prototype that’s light-years ahead of anything visual ebooks have managed. Just insanely clever and cool. Plus they’re insisting on markup (no PDF for them!), which gives markup a foothold in publishing it wouldn’t otherwise have.

In the meantime, reference and “variable content” material — that which is out of date the day it is produced — will increasingly be consumed digitally, continuing to create both a global readership and a publishing industry more and more accustomed to the digital world.

Yes. Right on. Trying to take the world by storm always was a dumb idea. This is how ebooks will find their feet. Slowly. In places where print is obviously, openly dysfunctional. And then we’ll make ’em better. Slowly. And then we take over the world.

It probably should not surprise anyone that Umberto Eco gets it about ebooks. Eco is a certified Smart Guy, after all.

Still. A lot of certified Smart Guys have utterly and completely failed to get it. So I still find it refreshing when someone—certified Smart or not—does.

I don’t wholly agree with the “nobody reads for pleasure on screen” dictum; despite my well-known bitterness about the Beach Blanket Bob mode of ebook marketing, I know some people will read for pleasure onscreen because I am one of them. However, Eco is careful to restrict his comments to past and present without trying to predict the future, so I can generally go along with him as far as he goes.

And he is oh-so-right about reference books. We can finally start to solve the thumb-in-the-index problem.

Some good words about libraries, too, with which I wholeheartedly agree though I don’t think they go nearly far enough.

(Here There Bee Rantage. Ye Have Been Warned.)

An occasional gadfly who aspires to be a thorn in my side emailed me in response to my plans to put together XML tutorials next semester. He just really wanted to let me know that whenever folks like me talk about this stuff, folks like his eyes glaze over.

Thanks ever so, dude.

The more so because you’ve never actually seen me give an introductory talk. CavLec mostly doesn’t count; when I write about tech stuff here, it’s at my own meagre tech level rather than designed for beginners, unless I say otherwise pretty explicitly. I am, as it happens, reasonably good at getting tech stuff across to newbies. Or I’ve been told so, anyway.

Well, I would have laughed it off if it hadn’t been for the notion, espoused by him and some others of my acquaintance, that really, books are simple creatures and they can be represented and encoded incredibly simply. All you have to do is just…

Excuse me a moment. “Just?”

Don’t anyone even come near me with that word in relation to book production until you’ve done your time. Copyediting, proofing, typesetting, design, art work, bindery, electronic publishing, SGML work, desktop-publishing software design, I don’t care—but damn it, do your time before you tell me that book production can be boiled down to “just” anything.

Nine times out of ten, these yahoos have utterly forgotten that there’s any book in the world more complicated than, say, a Robert Ludlum novel. (I don’t think these yahoos actually set foot in libraries, though I suppose I could be wrong—they could merely suffer from acute tunnel vision.) The rest of us don’t have that luxury, in particular because Robert Ludlum won’t sell ebooks. We have to sweat over math, art, indexes, tables, links, complex layouts, production workflows, metadata, non-Roman alphabets, digital preservation issues, and all that fun stuff.

And, I mean, it’s quite arguable that we’ve sweated this stuff in entirely the wrong fashion. That’s a hell of a long way, though, from the airy insinuation that sweating it at all is utterly unnecessary. Or “simple.”

This is, it seems to me, part and parcel of a miraculous human tendency to take the familiar for granted. Because print books are familiar, they must be “simple.” Well, I am here to say, it ain’t so. There’s four hundred years of innovation, invention, and refinement in the print book. There’s plenty of craft, plenty of human effort.

Now, y’all know me better than to think I think the print book is the pinnacle of information-disseminating achievement, never to be improved upon or even equalled. I’m just not stupid or clueless enough to dismiss the very real accomplishments and complexities of the print form.

Nor, to take the opposite extreme, do I think that either print books or ebooks are utterly beyond fathoming by the garden-variety human mind. I have a garden-variety human mind, and I don’t do so badly.

Bah. I have said all this before. I wish I didn’t have to keep saying it, but I’ll repeat as often as necessary, I guess.

Over at my fellow student Nichole’s there’s a comment discussion about the Amazon book-search gizmo.

Frankly, I didn’t (and don’t) think much of it, for the signal-to-noise problem pointed out in Nichole’s comments.

But then I had an idea that might just actually be good. What about back-of-book indexes? Wouldn’t it be wizard cool to be able to search those en masse, and have book titles along with the index entries (to check scope of references) returned?

(Actually, now that I think about it, I’d want number of pages in book, too. It isn’t necessarily absolute page count that helps determine whether a book has a lot or a little on a given subject; it’s ratio of index references to book length. And in a library, obviously I’d want this integrated with the OPAC, so I can have the call number and all that good stuff.)

I mean, when I go research something—and I’m sure I’m not alone in this—I scribble down a list of possible sources, go find ’em, and immediately check their indexes. This index-searching gizmo would cut a long way down on the time I spend on the first two tasks.

This would be dead easy to do; even converting indexes is relatively easy because they’re so structured. And it’d be an electronic window into print sources, something we all seem to want but aren’t sure how to build.

Did I just have a good idea? Somebody tell me what’s wrong with it, quick!

Alternately, somebody hand me some venture capital…

When I was thirteen or so, my dad bought the family’s first microwave oven. The fight that ensued sent my sister and me into hiding. “I will not have That Thing in My Kitchen!” my mother squalled, over and over again, while Dad tried fruitlessly to reason with her.

“Well, I won’t use it!” she declared, as her last word.

“Fine,” my dad said, nonplussed. “I never intended to force you. The electric oven isn’t going anywhere.”

So he used it. And he found out (eventually; my dad can be kinda stubborn) that it’s great for some things—for others, not so much. The regular oven really is a better choice for roasting chicken. And much later still, my mom found out about reheating leftovers. By now she probably uses the thing more than he does.

The regular oven is still there, and still gets used. But if she’d succeeded in shutting down the conversation between her and my dad, she’d never have known that the microwave, in its own way, was useful too.

Last week, in the discussion section of my virtual-collections class, the print-book snobs started in. I’ll never read an ebook (or, as they put it, “a book online,” which begs an awful lot of questions). What happens when the power goes off? (You can’t read a book in the dark. PDAs and ebook gizmos are backlighted. Wanna ask that question again?) The print book has Stood the Test of Time. (The ebook hasn’t had a chance to. Come back in four hundred years and we’ll revisit that one. Oh, and while we’re at it, how many incunabula are extant now, compared with how many we think were printed? New perspective on the much-decried data death, I suspect.)

Oh, and all that “technology” stuff—it’s bogus, it ruins our lives, we hates it, we does. (So much wrong with this I don’t know where to start. But defining “technology” might do as a jumping-off point.)

And the final insult: Computers will never replace the print book. Oh, come on, people, who said they were trying to? (Other than print journos looking for a catchy headline and snake-oil salesmen trying for another sale.) Honestly, the level of hostility toward print imputed to me as an e-text enthusiast by this sentiment verges on the personally offensive. Really it does.

Print-versus-e-text, to me, is exactly analogous to oven-versus-microwave. Print and e-text each have strengths and (little though those who would put the print book on a pedestal like to admit it) weaknesses. It behooves us to think about these things, experiment with them, find a good mix. We can’t do that if we dogmatically assert that e-text is bad.

To my shame be it spoken, I completely lost my temper. Foom. Gone. Oh, I’ve done worse on CavLec, and I certainly didn’t make the parenthetical smart remarks above, but it definitely wasn’t pretty. I have not read and do not intend to read the responses to my broadside (except for the one from the professor, which rather unexpectedly backed me up); if nothing else, CavLec has taught me when I really, really need to drop an argument.

What made me pop my cork, as I hinted above, was that people were shutting down substantive discussion of the relative merits of print and electronic materials, just as my mom tried to shut my dad down about the microwave. I don’t mind discussing electronic-text drawbacks. I do so regularly, and I learn from people’s objections also. But I discuss with a view toward making e-texts better, not demonizing them, certainly not wiping them off the planet. I ask, “Would it be better if ebooks…” rather than declaring “Ebooks don’t do X, so ebooks are eternally useless” and turning up my nose.

Now, I have my own technological blind spots. You couldn’t pay me to carry a cell phone; I don’t believe anybody needs to get hold of me that badly, and I strongly dislike the attention division that Rheingold rhapsodizes in Smart Mobs (which I am reading for my intro course). I do hope, though, I’m never quite so closed to discussion as my mom faced with a microwave oven.