AOL posts 20 million users' Internet queries

AOL posts 20 million
users' Internet queries
'Research' document later removed
after hundreds download data file

America Online posted on the Internet – for a brief time – all of the search requests made by more than half-a-million customers, setting users in a rage.

"The utter stupidity of this is staggering," one comment on the website TechCrunch.com said today.

The data released includes all the searches submitted by an estimated 650,000 users over a three-month period, the results of the search, whether the users clicked on the result and where it appeared on the result page

The file, was posted over the weekend and quickly removed, and TechCrunch said that means someone at AOL realized the damage that was being done and "is also an admission of wrongdoing of sorts."

Either way, the website noted, the information, a file of 439 megabits compressed and about 2 gigabits in standard file formation, now is available because of the estimated 1,000 copies made during the time it was up.

Later seekers of the posting were given an "Error – Bad Request" response.

While the AOL usernames had been changed in the file to a random ID number, TechCrunch said analyzing all searches listed by a single user often can lead people to determine the identity.

"The most serious problem is the fact that many people often search on their own name, or those of their friends and family, to see what information is available about them on the net," TechCrunch said. "Combine these ego searches with porn queries and you have a serious embarrassment.

"Combine them with 'buy ecstasy' and you have evidence of a crime," the website said.

For example, user Number 39509 searched for "oklahoma disciplined pastors," "oklahoma disciplined doctors," and "home loans."

Number 545605 searched for "transfer money to china" and "capital gains on sale of house."

The AOL file included information about 20 million total searches, and logged User IDs, questions, question times, rank of the clicks and destination URLs.

"The goal of this collection is to provide a real query log based on users," the page said. "It could be used for personalization, query reformulation or other type of search research."

The AOL page also provided two warnings.

"This collection is distributed for non-commercial search only. Any application of this collection for commercial purposes is STRICTLY PROHIBITED."

The second was: "Please be aware that these queries are not filtered to remove any content. Pornography is prevalent on the Web and unfiltered search engine logs contain queries by users who are looking for pornographic material."

One of the first comments in response to the posting was from a graduate student working on PageRank algorithms. She wanted to know if further details of the queries were available.

The posting comes just a few months after a judge rejected a blanket subpoena from the Department of Justice to Google, another major Internet presence. The DOJ had sought two month's worth of users' search queries, but Google resisted the subpoena, and Judge James Ware excluded search queries and limited the government's demand for URLs to 50,000.

At the time, Google called it a victory.

"While privacy was not the most significant legal issue in this case (because the government wasn't asking for personally identifiable information), privacy was perhaps the most significant to our users," the company's statement said.

"We believe that if the government was permitted to require Google to hand over search queries, that could have undermined confidence that our users have in our ability to keep their information private."

Google said the judge's ruling in that case meant "that neither the government nor anyone else has carte blanche when demanding data from Internet companies."

Microsoft also has considered releasing similar data to researchers, although not the part that would allow data to be associated with an individual user.

AOL's website did not address the posting.