New York-based blogger Jason Kottke noted in a post yesterday that since the transfer of power to Obama, the robots.txt file on the WhiteHouse.gov site . Search engines rely on robot programs to index content, and robots.txt is used to set limits on what is indexed. The Bush administration's "disallow" listings, which went on for 2,400 lines, were removed by the new administration and replaced by a that appears to have just a single entry.

Kottke, in an e-mail response to a question from Computerworld , said the material that was disallowed from being indexed by the Bush administration primarily consisted of "text-only duplicates of hundreds of pages. I would argue those text-only pages should be included in search indexes, but it doesn't look like there [was] a conspiratorial attempt to hide information."

Nevertheless, Kottke thinks that by eliminating the Bush disallow list on its first day in office, the Obama administration was sending out a symbolic message. "One of Obama's big talking points during the campaign and transition was a desire for a more transparent government, and the spare robots.txt file is a symbol of that desire," Kottke wrote.

The nearly blank disallow slate may not last long, though. "As the site grows and accumulates more content," Kottke added, "I'm sure they'll need to add to the directories and scripts that search engines shouldn't index."