Current opinions on challenge-response systems

On 12/15/06, dr <drdavidrossconsultant.com> wrote:

> CR systems seem great to the end user but put all the work on small business mail admins and
> almost anyone running a mailing list. When I was on my pool board, I'd get one or two of these a
> week. Someone would send in a request for directions to a meet, how to signup for the evening's
> activity, or whatever. I'd respond and come back at midnight or the next morning with a CR for them
> in my inbox. They never got the answer. And don't even ask about the hassles of people adding CR
> systems to their email and then yelling when they no longer got the announcements or missed
> something important. Is it really the job of a volunteer web master to spend an hour or more each
> week answering CR requests? And checking email every hour to make sure you catch them?

For the past eight years, I've worked various jobs in the anti-spam
industry. While opinions are widely varied, your assessment fairly
well matches what little consensus exists on the issue.

In addition to those issues, many CR systems generate a response for
EVERY message received. The problem with this is that at the moment
(according to some sources), upwards of 90% of all email is spam, and
much of that is sent with forged addresses in the From: and Reply-To:
headers. The upshot is that many CR systems improperly issue
challenges to addresses which either don't exist, or belong to
uninvolved third parties.

Even proponents of CR admit that this is an issue, and propose that
correctly implemented, CR systems need to be placed behind spam
filters, which reduce the amount of outright spam hitting the CR
system. In effect, CR is then , NOT an anti-spam measure in itself.

From the point of view of third parties, users of CR systems are
offloading the responsibility of vetting their email spam vs. ham upon
them. While on the surface, this seems to work well for the CR user,
in practice, random third parties get to decide what mail gets
through. I know several people who routinely answer misdirected
challenges, and many more who refuse to EVER answer an challenge even
if they *did* send the original mail.

Oh, and misdirected challenges *do* get reported to ISPs as abuse.

The bottom line here is that eventually running a CR system will cost
you legitimate email, and is likely to cause some spam to get through.
 On top of this, when running a CR system, prepare to field reports of
abuse forwarded to you from your provider. Hardly the panacea some
people tout it as.

--Brian McNett

>Oh, and misdirected challenges *do* get reported to ISPs as abuse.

It is shocking what gets reported as "abuse"... =\

Lists such as this one**, confirmation emails from ecommerce
transctions, personal correspondence, bounces, you name it. Reading
the "abuseforest.net" email account is a source of neverending
entertainment and a shocking assessment of the average intelligence
of the Internet user.


**thankfully Web Crossing embeds your email address in an encrypted
fashion, so if you report mail from tidbits.com to the responsible
ISP, (in this case us) we can unsub your account without having to
bother Adam. We can logically assume that if you are reporting list
mail as abusive, that you don't want to receive it anymore.


--
Chuck Goolsbee V.P. Technical Operations
_________________________________________________________________
digital.forest Phone: +1-877-720-0483, x2001
where Internet solutions grow Int'l: +1-206-838-1630
*** celebrating twelve years of service 7/12/1994 - 7/12/2006 ***
12101 Tukwila International Blvd Fax: +1-206-838-3749
Suite 410 http://www.forest.net
Seattle, WA 98168 email: cgforest.net

&
On 12/18/06, chuck goolsbee <chucklistforest.net> wrote:
> >Oh, and misdirected challenges *do* get reported to ISPs as abuse.
>
> It is shocking what gets reported as "abuse"... =\

Well yes, there *is* that (oh, Hello Chuck from a former customer, and
the abuse desk of YOUR network provider (one of them, anyway. I won't
say which here)).

> Lists such as this one**, confirmation emails from ecommerce
> transctions, personal correspondence, bounces, you name it. Reading
> the "abuseforest.net" email account is a source of neverending
> entertainment and a shocking assessment of the average intelligence
> of the Internet user.

I particularly enjoy the folks who report ecomerce invoices to us.
$large_candy_company gets several of these a day, swamping the
complaints about their open PHP script, and the stock spammer abusing
it.

We recently asked AOL to turn on a ;Feedback Loop" (FBL) for a couple
of customer ranges. The marked decline in average intelligence of
abuse reports purely as a result of the large numbers of AOL users,
is, at least in my opinion, completely offset by the *firehose-like*
quality of the AOL FBL. Complaints in quantity let us find spammers
INSTANTLY.

Still, most ISP abuse desk can tell stories about the entertainment
value of abuse complaints. One thing this job doesn't do, is give you
high hopes for the future of humanity. Egads, are they ALL idiots?
I'm constantly reminded of the saying:

"It is better to keep your mouth closed and let people think you are a
fool than to open it and remove all doubt." -- Mark Twain

We're currently seeing real issues with the numbers and types of email
bounces, such that I'm forced to consider complaints about bounced
mail (particularly bounces to third parties), as part of the problem.
I tend to lump them with complaints about emails from anti-spam and
anti-virus systems, and the aforementioned C/R systems. The main
culprits are qmail and (surprise!) Microsoft Exchange Server 2003.
Exchange Server can be patched, but alas, qmail, apparently takes far
more effort.

> **thankfully Web Crossing embeds your email address in an encrypted
> fashion, so if you report mail from tidbits.com to the responsible
> ISP, (in this case us) we can unsub your account without having to
> bother Adam. We can logically assume that if you are reporting list
> mail as abusive, that you don't want to receive it anymore.

Properly designed mailing list software *should* operate this way. I
get frustrated at customers who seem unable to wrap their minds around
this. Most of our customers, however, tend to be providers, such as
Digital Forest, who handle their own abuse, and are basically on top
of things. I haven't had to deal with Chuck in a business sense,
since I was a customer of Digital Forest six years ago. This is the
kind of situation abuse desks like.

--Brian McNett

>Well yes, there *is* that (oh, Hello Chuck from a former customer, and
>the abuse desk of YOUR network provider (one of them, anyway. I won't
>say which here)).

Which one? At the moment we have three. We'd have even more if they'd
stop going bankrupt and/or buying each other! ;)

Feel free to reply to that offlist.


>> Lists such as this one**, confirmation emails from ecommerce
>> transctions, personal correspondence, bounces, you name it. Reading
>> the "abuseforest.net" email account is a source of neverending
>> entertainment and a shocking assessment of the average intelligence
>> of the Internet user.
>
>I particularly enjoy the folks who report ecomerce invoices to us.
>$large_candy_company gets several of these a day, swamping the
>complaints about their open PHP script, and the stock spammer abusing
>it.

Bringing this back to C/R, this goes to show how futile such systems
are, the vast majority of spam is composed of completely bogus header
information, with forged everything, coming off either web-to-mail
scripts of webservers, or moire likely compromised Windows hosts on
consumer broadband networks. Sending them C/R mails is a complete
waste of everyone's time, cpu, and bandwidth. You annoy the people
you NEED to exchange email with, and fill the world's mail queues
with replies to nobody.





>We recently asked AOL to turn on a ;Feedback Loop" (FBL) for a couple
>of customer ranges. The marked decline in average intelligence of
>abuse reports purely as a result of the large numbers of AOL users,
>is, at least in my opinion, completely offset by the *firehose-like*
>quality of the AOL FBL. Complaints in quantity let us find spammers
>INSTANTLY.

True. My filter set in Eudora for handling SCOMPS is terrifyingly huge.

One of the several reasons why I hesitate to switch to any other MUA. =\



>Still, most ISP abuse desk can tell stories about the entertainment
>value of abuse complaints.

It makes for great "over a beer" discussion at netops conferences...
sort of like postal workers talking about dogs.


>We're currently seeing real issues with the numbers and types of email
>bounces, such that I'm forced to consider complaints about bounced
>mail (particularly bounces to third parties), as part of the problem.

Backscatter is easily 60% of my problem. We acquired an ISP in 2002
whose owner was a HUGE anti-spam crusader. As such his main domain
name is probably one of the most used domains for joe-jobs on the
whole Internet. sigh...


>Properly designed mailing list software *should* operate this way.

As should properly designed MTAs... at least when dealing with
backscatter. The problem is that legitimate mailers are a fixed
target. We are wedded to our CIDR netblocks and domain names, the
spammers however, shift, move, and change tactics with ease.

I predict that within the next five years, email will basically no
longer exist. Spammers will eventually kill the very thing that
provides their livelihood. A classic case of ecosystem destruction in
the name of greed.

Regards,
--

Chuck Goolsbee V.P. Technical Operations
_________________________________________________________________
digital.forest Phone: +1-877-720-0483, x2001
where Internet solutions grow Int'l: +1-206-838-1630
*** celebrating twelve years of service 7/12/1994 - 7/12/2006 ***
12101 Tukwila International Blvd AIM: chuckgoolsbee
Suite 410 http://www.forest.net
Seattle, WA 98168 email: cgforest.net