I decided to make a small thread on how I usually go about searching for information I shouldn't be able to find. Google is an excellent source for this and many people refer to it as Google hacking. With a well worded search you can find plenty of things.
#1 Robots.txt The way search engines work is by sending out search spiders or robots that crawl the web adding links to their database. So what if a web page owner wants to hide certain pages from google? They add a simple text document in their directory addressed to the robots telling them where not to go, and what not to catalog.
Examples: http://sprint.com/robots.txthttp://www.palm.com/robots.txtAlot of times these mean nothing to you, they are just things the website owner does not want cataloged by Google. Such as their images, specific in depth pages etc. Things that would just cause too much unwanted data to be returned by a search.
You can try to access the directories by adding them to the end of the URL you are currently at.
Examples:http://www.palm.com/assets/css/http://www.sprint.com/localbusiness/As you can see by going to these they often give you an error or redirect you somewhere else. This is because the company doesn't want you to see that entire directory.
These can be useful because if you have a certain filename you can most likely access that.
#2 URL swappingSo if a site follows a common scheme for their site and the post something early but don't reveal it you may be able to find it.
Example:http://nextelonline.nextel.com/NASApp/onlinestore/en/Action/DisplayPhones?phoneSKU=PTR800HKThis will lead you to the Palm 800w info page. Because the SKU at the end is PTR800HK Now from working in retail I know how to figure out what a lot of SKUs means I think that:
PTR = Palm Treo, 800 = 800w, HK = I'm not sure
We can see that the Treo 755p link is as follows:
http://nextelonline.nextel.com/NASApp/onlinestore/en/Action/DisplayPhones?phoneSKU=PTR755PHKSo we now kind of understand Palm's and Sprint's page setup.
If we knew the SKU of the Pre and Sprint has posted that info early, which I doubt they have, but may in the future. We could just change the SKU and access it.
#3 Google trends etc...http://www.google.com/intl/en/press/zeitgeist/index.htmlThis is google's press page you can search for trends, compare search terms, find trends for the year etc...
#4 Directory Search:So you found a nice directory and you don't have access to the directory listing but you can reach files within but would like to see the listing anyways. This is where Google will help.
Example you want to see everything within the
http://www.palm.com/us/ directory.
Search google for:
allinurl: http://www.palm.com/us/
Click "repeat the search with the omitted results included"
And you will see these results:
http://www.google.com/search?q=allinurl:+http://www.palm.com/us/&num=100&hl=en&safe=off&client=firefox-a&rls=org.mozilla:en-US:official&hs=fNv&filter=0And there you have their directory listing.
I will add more later but for now I am going out with some friends.