Web Design Forums

Mupolan · Joined: 30 May 2006 Posts: 101

Hey, does anyone know how dogpile is legal, since google doesn't allow itself to be spidered?

autos06 · Joined: 13 May 2006 Posts: 123 Location: New York

Mupolan, it just does a HTTP request to google?
Not that hard to pretend to be a browser :d
And holy crap http://www.dogpile.com/info.dogpl/search/web/foo%252Bbar/1/-/1/-/-/-/1/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/417/top/-/-/-/1
Little much with the URL parameters Neutral

Mupolan · Joined: 30 May 2006 Posts: 101

Google doesn't allow itself to be spidred
Could't they stop dogpile?

autos06 · Joined: 13 May 2006 Posts: 123 Location: New York

What do you mean "doesn't allow itself to be spidered"

Mupolan · Joined: 30 May 2006 Posts: 101

Robots.txt

autos06 · Joined: 13 May 2006 Posts: 123 Location: New York

So spoof the useragent?

Mupolan · Joined: 30 May 2006 Posts: 101

They could sue dogpile

autos06 · Joined: 13 May 2006 Posts: 123 Location: New York

How so?

Mupolan · Joined: 30 May 2006 Posts: 101

With lawyers

autos06 · Joined: 13 May 2006 Posts: 123 Location: New York

:d
They put their content on the internet, public domain

Mupolan · Joined: 30 May 2006 Posts: 101

That's not how it works

autos06 · Joined: 13 May 2006 Posts: 123 Location: New York

I doubt they'd get anywhere with a law suit

Mupolan · Joined: 30 May 2006 Posts: 101

Ok, you have no clue

autos06 · Joined: 13 May 2006 Posts: 123 Location: New York

Mupolan, there's also the google api Wink

Dogpile could just be doing searches with that :d

Peter Pitt · Joined: 20 May 2006 Posts: 114 Location: Egypt

Mupolan,sue them for what? /Linking/?

autos06 · Joined: 13 May 2006 Posts: 123 Location: New York

Mupolan, http://research.yale.edu/lawmeme/modules.php?name=News&file=article&sid=1725 interesting read
And where does robots.txt become legally binding?

Mupolan · Joined: 30 May 2006 Posts: 101

Look up ebay's "virtual tresspassing" lawsuits
They stopped a lot of meta auctions ites
The web used to be filled with meta auctions

autos06 · Joined: 13 May 2006 Posts: 123 Location: New York

The concept is totally flawed still
A "user" could be doing the search, saving the google page and then having it parsed by the application
The "user" is not blocked by the robots.txt, just robots.
S/he was saying that google doesn't allow robots to index it
But if it wasn't a robot that grabbed the page, then the robots.txt doesn't matter
"only people 7ft tall can read this sign" but if I get someone who is 7ft tall to read it for me and then describe it, the height restriction is meaningless Smile

froomzer · Joined: 23 May 2006 Posts: 148

Since when is robots legally binding?
If you have a page on the web
It's accessible
If you donct' want it accessible
DON'T PUT IT ON THE WEB

autos06 · Joined: 13 May 2006 Posts: 123 Location: New York

Exactly
Rhett, the lawsuit looks hilarious
In blocking the practice at least until the case comes up for trial, U.S. District Judge Ronald Whyte ruled that Bidder's Edge had "deprived eBay of the ability to use...its personal property for its own purposes."
I think the biggest issue is that the officials in office have no idea about technology, so we end up with silly laws and suits, and patents on technologies that shouldn't ever have been given