RHF image posting feedback & FAQ

On June 17, 1999, we did a test posting of an image (cartoon) to rec.humor.funny. The posting was a multipart/alternative, with plain text including a link to a Darth-Darth Binks web page and HTML that would offer both that link and the direct display of the image for the half of the net with an HTML-handling newsreader.

In addition, readers were asked to provide their opinions on the matter in a survey.

Wow! It generated a great deal of response, as well as many questions. The answers go to the very heart of the nature of USENET itself and all the controversy that surrounds it.

Opinions were decidedly mixed and widely split. In many cases there is no clear consensus of reader opinion. This document was also posted Jun 22 to rec.humor.d -- that is where you should post any comments, rather than E-mailing.

As expected, about 50% of readers saw the image immediately in their newsreader (either Outlook Express or Netscape, the 2 most most popular USENET newsreaders.) The others found their way to the link either in their newsreader or in a browser.

Around 1600 took the survey in the first 4 days (including a weekend.) The survey was of course self-selected, which limits its value.

Executive Summary

1. Opinion is divided, but a majority would like to see more images if funny.
2. Images, if presented will be plain text postings with URLs of a web page with the image alone on a line so almost all users can easily click or cut and paste.
3. Images will in any event be small, in the range 10K to 40K.
4. There is much division about whether it's better right in the posting rather than a link, but it will be a link for now.
5. Netscape is the most popular newsreader for rec.humor.funny readers, with TRN 2nd. (Unlike the broad net where MSIE/Outlook Express is second.)
6. Lots of philosophical discussion on the nature of USENET below.

Viewing the Image

Of those surveyed, 30% said they saw the graphic directly, 10% could immediately click-through to it, and 34% could do a quick cut and paste into their browser. (X windows users may know that you can triple click on a line in an xterm with a URL, then move the mouse to Netscape and middle-click to go to a URL found on a line by itself.)

10% of users reported they use shell accounts, 3% said they read offline (this number is no doubt skewed low). 3% had the newsreader invoke the browser immediately, and other results accounted for under 5% of users.

In other words, most people -- at least those who could easily take the survey and wanted to -- saw the image without much difficulty.

Very few reported a failure, in spite of the bugs described below.

Newsreader Stats and abilities

I asked people to name their newsreader. Kjell Irgens calculates great stats on newsreader usage (based on posting). These give us some idea of popularity of newsreaders. Many have been stunned to see MSIE/Outlook Express and Netscape account for over 50% of postings on the net at various times, though recently they are down to 45%. While one can make some debate that perhaps all the lurkers are running other readers, somehow I doubt lurkers vary by more than a moderate margin from posters.

RHF readers do vary, however. In the table below Netscape is the big winner with 22% of respondents, and Outlook Express only has 11%. TRN has a hearty 15%, and TRN prior to 4.0 doesn't even show up in the posting stats, but I have long suspected it to be a close competitor for Agent for the #3 spot netwide.

Most people have a newsreader that handles basic MIME today, and almost everybody has a newsreader that can handle MIME if they upgrade to the latest version. MIME was written into the unofficial USENET newsreader spec back 7 years ago.

Most of the other popular newsreaders -- Agent, Newswatcher, gnus and others offered clickable URLs from the plain text or browser invocation from the reader, and the rest, TIN, TRN etc. offered easy cut and paste to those who run a windowing system and, like most, have a browser open.

It's hard to say how many people read offline, as many of them would not bother with the survey. However since most groups (other than low volume rec.humor.funny) are so high volume as to be impractical for offline reading, the number of such people is dwindling.

Web Browser

Of those who came to the survey, I recorded their web browser, and the vast majority (72%) came with Netscape. Only 23% used IE and 2% used lynx. This does not match ordinary web surfing. USENET or RHF readers must not like Microsoft for some reason. :-)

How did they feel about images

Evenly divided, but with only a 20% feeling they did not belong in the newsgroup at all. That's very different from the sense you would get from reading the comments. Those who were against them left far more comments than those in favour.

Complex Question of Inline (direct) images

Many users were confused, but the experiment only included links to my web site, not the actual images themselves. Users of Netscape Newsreader or Outlook Express saw the image directly because the link was an <IMG> tag -- something I probably won't repeat, as detailed below.

However, I also asked about using links vs. putting the image directly in the posting. Boy is that controversial.

Putting the image directly in the posting is actually the "USENET way" of doing binaries. It takes advantage of all that is good about USENET -- fast local access to what you want to see or read. It distributes the image and thus requires no central web server (which may be slow, far away or down). It also makes the image available to those without internet connections, or those who read offline with a capable reader. That's the USENET way.

Sounds great, right? But it also means pushing the image to those who read offline whether they want it or not, and also take time and bandwidth for those reading via NNTP over a modem without giving them the option of easily aborting or making a 2nd decision to see the image. It's also slightly inefficient because MIME encodings take 33% more space vs. sucking from the web, though modems re-compress this overhead.

The porn/binaries groups have also made most readers able to handle this.

Those who want the image get the most benefit from seeing it right in the posting. Those who want to choose (other than based on the subject line or other headers) prefer a click. Those who hate the images hate inline distribution even more (you should read the notes!)

The conclusion is that for now, use of a web link is best. That does mean requiring a fast, high capacity, reliable colocated web server, which we do have. The load actually balanced out pretty well so it was well within the capacity of the server. If USENET groups got millions of readers all viewing something at once, the inline method's efficiencies would win out.

The slim majority as you can see prefer a link, but a surprising 37% would like to see inline (with a link as well -- that's easy.)

An article on issues in image formats is available.

Darth-Darth

I also asked if you liked my cartoon. I knew the cartoon was reasonably funny because I made a T-shirt of it and got great reactions at USENIX, though of course there is no pleasing everybody. Comments ranged from "made my day", "my son hasn't stopped laughing", "you make it all worthwhile" to "I wish you had picked something funny as your first experiment."

Frankly, since -5 meant "least funny thing you have ever seen" I expect some of those votes came from people just pissed off at the idea of images.

Answers to various comments

Don't post binaries. They eat up my disk space!

A surprising number of the comments were from users concerned over disk space usage, bandwidth and other issues that implied that they thought the image had been posted inline, as a MIME format binary.

It wasn't. It was stored on a web server and fetched as a link. There are many reasons why doing it inline is actually the right "USENET way" of doing a binary, there are enough people who don't like it at this time that it probably won't be done, even though rec.humor.funny, with its extremely high readership percentage on articles, is best qualified for such activity. The antipathy is just too high, so it's not likely to be done.

Binary filters will start killing rec.humor.funny

Well, since only a link was posted that's not likely. Killers that kill HTML or multiparts might filter it. However, I strongly feel that while a site can filter as it likes, most of these filters are there to stop abuse and violation of newsgroup rules, and they should never interfere with a properly approved posting from the moderator in a moderated group.

Existing ones, like bincancel, tend not to deal with postings under 100K anyway. In spite of some beliefs, binaries do not just appear in alt.binaries.

Be sure to label any images

Of course. This image was well marked. If images were to become regular, a standardized label would be developed for those wanting to killfile.

The MIME was broken

Yes, I made an error in the MIME because the automatic generators make ugly articles, and I did some things by hand and got the final delim slightly wrong. Amazingly, almost all the tools handled it anyway! (I had only tested in a few, and they worked.)

The URL for the survey in the plain text form was broken

Mea culpa. One nice you have to give the web is the ability to fix things after the fact. Within 2 hours a redirect was in for the broken URL to make it work.

Why did the two variants differ? What about spoilers

Those seeing the HTML form of the posting were likely to see the image immediately, so some text came along with it to supplement the image.

Those reading the plain text had to do something to see the image, so the text, which might spoil the image, was not included.

I apologize for not including a spoiler warning. The spoilers in this were pretty minor but I still should have done it.

Doing your survey on the WWW will bias it.

This is true. However, E-mail surveys just are not practical. I have done them. This survey got 1600 responses, and only a web form is practical if you want responses in a form you can do statistical analysis on. People just won't follow the rules for plain text forms, sadly.

As for the bias, the strong impression I have is that web access has grown extremely ubiquitous, and that while there are certainly USENET only people out there, almost everybody by now has arranged quick and convenient web access for themselves, by lynx if nothing else. So the effects of the bias will be limited.

Why not create a rec.humor.funny.images subgroup?

This was by far the most common suggestion of those who had a comment. There are a couple of answers to it.

First of all, it is not very likely at this time that the number of images that would come here would be anywhere near enough to create a subgroup. Especially if they are presented at links, skipping the few images would be such a little effort as to make it not worthwhile. There might be one or two images a month.

However, I also feel that it is entirely the wrong idea to create separate groups and hierarchies based on format rather than subject matter. It's a commonly proposed idea, but not an idea that works long term. Newsgroups are groups of people with a common interest. In the case of RHF, it's the comedy we select. One group for those who want HTML vs. one or those who want plain text vs. one for those who want the images vs... It's an idea that's simply wrong to me.

The reasons that people support are primarily these. In many images groups, the images often are the subject matter, so the groups make sense. In some cases, the images are so voluminous that it is desired for some sites (though it has always been a minority) to have an easy means of avoiding them. That's a bit of a kludge, but was felt necessary. The truth is that it makes more sense to have the feeding tools be able to filter based on size, or content, than to use the newsgroup mechanism to handle every filtering decision. However, the tools didn't do that, so people used what they had.

People are also worried, justifiably, about abuse, both by deliberate abusers and newbies. These issues don't apply to moderated groups.

And the "anything but 80 column monospace text is evil" lobby is vocal.

However, rec.humor.funny is about comedy, and sometimes comedy comes in other forms besides words. If it's an image that matches the mission of rec.humor.funny, it belongs there, not in alt.binaries.

Images are so huge! I pay by the minute! I will need to killfile!

The image in question was 28K if you made the web connection to get it. About 9 seconds on a typical modem. There have been many text items in the history of RHF that have been larger. Today's bandwidth abilities are just orders of magnitude beyond those of the past. While there are a very few people left with very slow or expensive connections, it would be an error to say that USENET can never go beyond its lowest common denominator. (However, articles should indeed be tagged or otherwise set out to help those users wherever possible.)

Most cartoons and images for jokes can fit easily in 10 to 40KB.

I was astounded at the few who said they would, if RHF contained images, need to remove it from their feed or filter it. Even if the images were inline rather than just links, their volume in RHF would still be dwarfed by the average low-volume unmoderated text newsgroup. I can only put this sort of comment down to emotional reaction, not rational fear of high disk and bandwidth costs.

USENET is for text only, outside alt.binaries

Many even prefaced by saying, "I believe in the old school." That one shocked me. In the days before the web, groups with source code and binaries, meant primarily for computer rather than human consumption, were among the most popular. For information, I went back 10 years ago to the newsgroup popularity rankings for June of 1989. Here are the top 25.

1. rec.humor.funny
2. news.announce.conferences
3. misc.jobs.offered
4. comp.sys.ibm.pc
5. misc.forsale
6. alt.sex
7. rec.humor
8. comp.sources.unix
9. comp.graphics
10. news.announce.newusers
11. comp.sys.mac
12. rec.arts.movies
13. misc.wanted
14. comp.sources.misc
15. news.groups
16. comp.unix.wizards
17. comp.lang.c
18. comp.unix.questions
19. soc.culture.china
20. news.misc
21. comp.binaries.ibm.pc
22. comp.misc
23. misc.jobs.misc
24. alt.sources
25. comp.arch

As you can see, in the "old school," groups with other than plaintext discussion were among the most popular, and were not relegated to 'alt.' They were also among the highest volume.

Why? Because USENET, as a distributed medium, was actually great for large binary files and source. You got it on your local system. You didn't need a connection to the internet or ftp, and it was faster even if you did have one.

Now some might say that this doesn't bear on the matter today, in the world of the web, but the old school was certainly not "all text" and USENET was not designed "exclusively for text" as some felt.

This will just encourage newbies to use HTML and images

While I see the validity of this concern, I feel from a philosophical standpoint that we must not fear (especially in moderated groups) using new tools just because others might abuse them. You won't get anything new if you live that way.

Multipart/alternative is bulky and bad

While it is very inefficient, it was the only format for the experiment I wanted to do (which included testing how HTML browsers function and how many people use them.) For better or worse, Multipart/alternative is the format the MIME group developed, and USENET has adopted, for format migration and handling different browsers. I don't think it is the best and in fact designed my own better solutions. The details of this can be found here.

However, since there are problems with mult/alt, in the long term I think that the best way to provide a link is with a URL on a line by its own. Many readers pop that up as a link, and almost everybody can easily cut and paste it.

I do feel, however, that it's a kludge. Guessing links out of plain text is a really bad software design. Formal specs are much better. If we can get my OOB HTML working, that's the way to do it.

The inline image scares me. I don't like the automatic nature

Well, I agree there are concerns here. Obviously in an ideal world, you want to present a clean interface, and you would want to just automatically show the image. Users with concerns would just turn off auto loading of images. However, for now people are too worried. Some said they couldn't turn it off (that's a buggy design, I feel) and some worried their boss looks at their web fetches but not their newsreading. Some said it's visually more obvious they are reading jokes at work!

So for now, this will be avoided.

Will graphics be tasteful?

RHF does carry dirty jokes. It might carry a dirty cartoon, but there will be plenty of warnings if it does, including before click-through.

This is the start of the slipperly slope of an HTML USENET

You'll ruin the text purity of USENET

Here's where I'll get really philosophical.

USENET is not about text. It's about distributing information to a community. It differs from the web in many more important ways than just the fact the web uses HTML and USENET (outside binary groups) uses mostly plain text.

First of all USENET is serial. It comes one message at a time, and is meant for tools that track the flow and show you what's new for you, much like mail. It is read in a stream, not browsed like the web. It's ideal for discussion both discussion and publishing of news and other serial information.

USENET is distributed. You read it from a local server. In the original vision, you read right on the news server or over a LAN. Access to material was fast, and didn't depend on any remote system being fast or even working at all.

Newsgroups are communities of interest. What matters on USENET is what the readers want to read. It was named "user's network" for this reason. This doesn't mean that if some people don't want to read something you can't do it. Nothing could ever be done if that were the case, certainly nothing new. But if the readers want it, be it picture, text, HTML or even video, it's right for USENET.

USENET pre-delivers material locally as indicated. The efficiency of this predelivery doesn't depend on the content. To it, bytes are bytes. If there are readers for an article at a site, it makes sense to deliver the article to the site in advance whether it is 1KB or 1MB. (Some might say it makes more sense to deliver it in advance if it is 1MB because fetching 1MB over the net when the reader requests it is slow.) There are many arguments that for very large and very popular items, USENET is actually better than the web. It's effectively a very stupid "optimistic cache." If an article doesn't have a reader, it's inefficient to deliver it no matter what size it is.

None of this says that USENET is nothing but 80 column plaintext. What 80 column plaintext is is simply something everybody can display. And being limited to it avoids, in a fairly drastic way, the abuse of pointless features often seen in web pages, especially as HTML went astray in version 3.

There is much in USENET that isn't represented in 80 column monospace text. Paragraphs for one, so that windows can be different sizes, and people can read it on smaller screens. Headings. Small amounts of emphasis. Formally defined links to useful resources and other USENET articles. Complex text inclusions with references to prior articles and their authors. Personal signatures that shouldn't be included in followups or search engine indexes. And pictures -- the source of most of the net's bytes today.

We're so scared of how HTML has been abused that USENET has remained in 1981. Even the online services of the 80s knew where the paragraph breaks were because they had users on Apple 2 computers with 40 column screens and needed to fold the text for them.

We post our FAQs to the net but these days we mostly write them in HTML or other richer formats, fill them with links, and post a dumbed-down version to USENET. Yet any suggestion to go beyond 80 column monospace is met, by some, with a cry that the way to do that is to start from scratch, creating a new hierarchy from the ground up to contain whatever the new proposal is. Creating a whole new net like that is hard. I'm one of the few people to have done it. Many others have tried it to failure or very limited success. People don't want to pick their newsgroup based on how the articles are formatted; they want to pick it based on the topic, and the people in it.

In almost every group people will loudly state that anything but plaintext is absolutely verboten, even if the volume of it would not significantly increase the group's traffic. Some even say, "If you have a picture relevant to this group, post it in alt.binaries.misc" as though people would see it there. As though the format of a message is more important in deciding where it goes than whether it is on topic for a group.

Sometimes, and a cartoon is an example, a picture tells the story better than words will. USENET is not a group of people dedicated to telling their story in monospace text, it's a group of people who want to communicate in a serial fashion, over a distributed network.

That's not to say that there should be a net full of postings that half the users can't handle, or which look ugly to most of the users. It's to say that a means for technical migration within the existing structure is the right idea. To suggest we figured it all out about message format in 1981 is ludicrous.

Is this the thin end of the wedge to the invasion of the web onto USENET? Not if we do things right. Rewards come only with risks.

So When?

Some say -- sure, having paragraphs and links and a formal spec would be great, but not yet. Users can't handle it. The question then is when? Don't say we can't do a migration, say how we can. I did, by designing two formats, one called Proletext and the other called Out of Band HTML

I think the latter is the right format for migration, though it is as yet unsupported.

However, even without those formats, today almost all users can handle MIME, and most can even handle HTML. They certainly can handle MIME if they get the latest version of their newsreader, and new USENET specs being drafted will encourage MIME, though not HTML in particular.

But the interesting question is, what percentage of users' tools need to support a new format -- any new format -- before it can be used? If 50% or 90% is not enough, what is? This question is not specific to MIME or HTML, whatever your opinion of their merits might be, it's for any new feature or format. Even if you don't feel we need any new features at this time, are you bold enough to say that we never will?

I think it's past time. USENET has been losing mindshare for years now. People are regularly moving to do their online discussions in web boards that aren't nearly as good as USENET, but can present users with an easier to learn and more appealing user interface. DejaNews, longtime bastion of support for USENET in the web world, decided to drop the "news" from its name and de-emphasize USENET. USENET volume grows but not nearly so fast as participation in the net does. Other sectors of the internet change completely in a year -- USENET has had almost no new features to its basic formats in a decade, and relatively few since B news in 1982.

(And those new features don't even fully work. Supersedes has a bug. References chains are often broken. MIME still remains controversial.)

The answer is not to flood USENET with MIME, HTML or Images. it's to examine what it is that can allow USENET to evolve, as its users want it to. It's been unable to do so up to now.

Newsreader Stats

Here were the stats people provided on what newsreader they read the image article with.

A small number of other newsreaders were listed, they were not significant in count.

RHF Home | Current Jokes | Best of Jokes | Search