Peachpit 0.5
A censorware-spider tarpit

This is peachpit, an automated tarpit utility aimed at getting in the way
of censorware spiders.  Peachpit provides a minor impediment to the functioning
of selected spiders, while presenting no obstacle to any other client.  The
agent selection may be made highly specific and constructed to avoid posing
any difficulties to any other spider or web agent.

The agent matching rules supplied are based on what little data is available at
time of writing on censorware spiders -- the manufacturers of censorware
products have historically been quite reticent about yielding any details about
the workings of their software, often going to unethical extremes to avoid
disclosure (see http://www.peacefire.org/ for lots of gory details).
Corrections, additions or similar would be very welcome if you have knowledge
of additional censorware spiders operating on the network or how they may be
identified.  Webserver log entries would be especially helpful.

Installation details are in INSTALL.

What is tarpitting?

Tarpitting is a tactic developed for use by mailservers in dealing with
spammers.  It's the process of gradually slowing down the response time of a
TCP connection or of data supplied through same.  Data is delivered
sufficiently to avoid a remote timeout, but so slowly that no useful
transmission may occur through the connection, tying up an entry from the
remote connection pool and generally providing a speedbump.  In dealing with
spammers, ultimately the connection is disconnected before mail is received,
thus obliging the spammer's MTA to resend (or, more commonly, give up).

Peachpit applies the tarpitting principal to dealing with hostile web spiders;
for this purpose, the censorware spiders.


Censorware?

This is a long and complicated issue, and you really should make a concerted
effort to understand and think all the way through it before deploying
peachpit.  Peachpit is a dirty trick -- it runs contrary to the principles of a
responsive, well-engineered network.  That said, so does censorware; as in the
battle against the spammers, we've found that playing by the rules makes for a
losing game against damage to our network.

You really should establish your own position, but mine is summarized thusly:

1. I don't believe in censorship.  At all.  I believe no person has a right to
interfere with the acquisition, consumption or untainted evaluation of
information or media by another person.  I have no sympathy whatsoever for the
agendas of religious organizations who have repeatedly attempted to apply their
own standard of morality to others, in this instance with respect to what
information others may be exposed to or produce.  I also suggest that the
United States' vast preoccupation with, and paranoia about sexuality is a
product of a lot of unintentional social messes and misguided religions, and
most governmental efforts to restrict pornography are simply the thrashings
about of these same influences.

2. Without actually agreeing with the efforts of censorware authors in any way,
I will for the moment concede that parents might wish to provide a more
constrained informational environment for their children.  I haven't decided if
that's right or wrong (I'd feel it was wrong if done to me, and I will not do
it to my children), but for the time being I won't interfere.

3. I see no way, based both on my own experience developing autonomous web
agent software and the actions of censorware authors, that web spidering will
anytime soon achieve an acceptable level of accuracy.  Every censorware
producer has claimed that their blocked-URL list is universally human-audited
to avoid inappropriate blocks.  In every case, they have been demonstrated to
have been either mistaken or lying -- by blocked pages that could never have
been rationally conceded as apropriate for their blocking categories.  The
censorware companies have demonstrated no willingness to adjust or cease
depending upon their spiders.

4. The censorware producers have themselves acted in extremely unethical ways
in the promotion of (see #1), and defense of their products.  Most have,
through their actions, demonstrated how hazardous they are in a free society.


$Id: README,v 1.1 2000/03/28 06:46:02 aqua Exp aqua $