The prices that Gainesville Coins has been serving to my web crawler differ from the prices they serve to a normal web browser. As a result, Gainesville Coins appeared to have the lowest prices in the price-comparison tables, though in reality, they did not have the lowest prices. This behavior compromises the integrity of this site. Therefore, Gainesville Coins will no longer appear in the price-comparison tables.
It came to my attention when I received the following comment on the ABOUT page:
Your quotes for Gainesville Coins is always the lowest AND it is always wrong!
When I looked into the issue, I found that the commenter was correct, so I immediately removed Gainesville Coins from the price-comparison tables until I could figure out what was going on.
My initial suspicion was that Gainesville Coins had changed their site’s layout in way that confused my web crawler, but this was not the case. In fact, the prices that my web crawler found were nowhere to be found anywhere on Gainesville Coins’ pages. After many hours of investigation, I finally realized that Gainesville Coins appeared to be engaging in a practice known as cloaking. Cloaking means that the content they present to a web crawler is different from the content they present to a web browser.
Whenever my crawler asks a precious-metal web site for a page, it identifies itself as comparegoldandsilverprices.com. This is standard procedure, so PM dealers can identify my server by its name and IP address. Not all web crawlers identify themselves as I do. In fact, many identify themselves as Google or another search engine, to fly under the radar, and it doesn’t take mad hacking skills to do this. Most likely, Gainesville Coins used my identifying information to trigger their own scripts, which would give false prices to my crawler. In other words, whenever my crawler asked for a page, the Gainesville Coins server would know that it was my server making the request and return a page with false prices.
To confirm the suspicious behavior, I created a simple script that fetches a page from Gainesville Coins without performing any other operations. The script simply requests a page from Gainesville Coins, like any ordinary browser would, then displays (echoes) the page on the screen. The only difference is that styles from external CSS files are not loaded, and that’s why the pages are styled differently in the screen shot. I ran the script on two different servers:
- my normal server (comparegoldandsilverprices.com).
- a control server unknown to Gainesville Coins.
Then I used a web browser to simultaneously open three tabs (from left to right in the screen shot above): one tab with an original product page from Gainesville Coins, one tab for the control server and one tab for my server. When comparing the fetched pages to the original page from Gainesville Coins, I could clearly see that the prices on the page from the control server matched perfectly with the original page from Gainesville Coins. However, the prices on the page that my server fetched did not match. Instead of the actual prices, false prices were displayed, all of them discounted 3%. In other words, when an unknown client made a request, Gainesville Coins would display the actual prices, but when my server made a request, false prices were displayed.
|Test Results||Bank Wire Price||Credit Card Price|
|Gainesville’s page viewed directly in a browser||25.56||26.31|
|The page Gainesville served to my anonymous control server||25.56||26.31|
|The page Gainesville served to comparegoldandsilverprices.com||24.79||25.05|
I cannot recommend PM dealers that engage in cloaking. I will no longer feature Gainesville Coins in the price-comparison tables. You can still read and write reviews for Gainesville Coins.