I would like to discuss the boundaries of SEO for persons who generally believe in abiding by TOS rules.
I am hoping that this will be a discussion and not just the usual on this forum where I post a couple of questions...a few respond with very short snippets of input...and that's that for the thread (that does not always happen mind you but it does seem to happen fairly often which is why I generally hang out at other forums).
Generally speaking I believe in abiding by TOS rules and I do not encourage breaking them. Mainly from an ethical even a moral standpoint with respect to doing what is right by both God and man.
But...something I have been thinking of lately is that this places me in a disadvantaged position while pursuing internet marketing success.
For example...
Google's TOS says explicitly that no automated means can be used to access their service. What this means according to the common definition of what "automated" means is that one cannot use ANY software that automatically interrogates Google for ranking positions.
If one is going to keep track of where one's site ranks for a given phrase at any given time the only option available to a person who wants to keep track of their ranking is to do it "manually" by doing a manual search every time they want their rank.
While everyone else violates the Google TOS left and right using such popular software as Niche something or other, Samaria whatever, and any number of such software I am stuck having to manually look up my site rankings every single time (and that's not the half of it...I also look up Google number manually when doing keyword research).
That's time consuming and if time is money I am at a distinct disadvantage when it comes to making efficient use of my time. All because I want to abide by a TOS.
Now don't get me wrong. Time disadvantage or not...abiding by agreements I make is an absolute principle I live by and advocate.
The problem is that Google itself does not enforce their TOS in such a way that it levels the playing field between those who would violate it at will and those who would not.
The other day I was riding a public bus and was presented with a similar dilemna. The rule on the bus said "No Eating or Drinking". Clear. No question about what it means.
I had a banana I wanted to eat. So I went to the bus driver and asked if I could eat a banana and looking at me somewhat surprised as to why I was asking her...she said that would be fine.
I sat down and ate my banana at which time it dawned on me that this was very much like the Google TOS situation.
There is a rule. No automated access to Google. Yet everyone who uses the aforementioned software does it.
So where does that leave me?
Do I not eat my banana because the rule says not to? Or do I just eat my banana with an awareness that the bus company does not enforce it's own rule and that everyone else who is hungry while riding a bus...well...eats their bananas without a second's hesitation?
I can write scripts to automate access to Google and save me a huge amount of time. Such access is not at all denied me. I can do what others are doing using their own software.
Should I do so? Should I violate TOS rules, the letter of such rules, if the rule maker itself is not enforcing them or can enforce them?
Where does one draw the line on such TOS violations? Do we throw all TOS obedience to the wind and just do whatever we want or can get away with in the realm of internet marketing?
Your thoughts?
Carlos Do you need more than what Google offers within their Webmaster Tools in terms of tracking your placement in the SERPS? And as far as keyword research, have you not found Google's AdWords Keyword Tool to be effective?
This might be the happy medium for you to stay within TOS while still saving some time getting the information that you want. I'm on the same side as you when it comes to following TOS. Those that show flagrant disregard for them tend to be shooting stars... they look good for a short while and flame out when they get caught and banned.
That said, on the topic of checking your own PR I honestly dont think using software to check it on occasion is what G was trying to avoid when they wrote that rule. Sometimes rules are written simplistically because brevity aids in the probability they might get read, and as a result something that isnt meant to be verbotten appears so. My suspicion is that such a rule is intended to disallow bots from running ongoing PR checks for the purpose of populating fields in websites. What you're wanting to do IMO meets the spirit of the rule though by a strict reading appears to be a breach.
That's just my opinion of course, and I'd love to see someone ask google that question in their webmaster section and see what they say. Thanks for suggestions Cricket but I think I probably didn't explain what I meant well enough.
Let's say that I am targeting a given phrase...let's see...how about "black tuxedos". I create a site page about black tuxedos. I want to see how well my SEO efforts are doing by periodically seeing where that page is ranking in Google for a search on "black tuxedos".
When I first create the page and Google detects it I rank at 123. I make a few changes on-page SEO wise and after the next Google crawl I move up to 87. I make some more changes and then move up to 53, then 15, and so forth. Or I might move down if my changes were not good.
So I am tracking my Google position over time.
Now I know that some will say that such tracking is useless and so forth but without getting into the relative merits or not of such tracking that is what I am doing. It is the best way that I know how to determine if my SEO efforts are meeting with success or not.
Now I am targeting a multitude of phrases with various keyword phrases at various domains.
To keep track of my positions for each phrase that I target is getting to be quite a bear.
That's what I meant cricket.
Respecting what Google offers through their Webmaster Tools I do not believe that such gives me the ranking positions that I am talking about. I could be wrong. I'll have to check into that but I doubt they provide that.
With regard to Google's keyword tool yes, that is what I use. But...there is NO way that I can do manually what I can do under automated script control to get the numbers from Google that allow me to pick out a very few possibly good keyword phrases from among hundreds.
To do it manually is a very time consuming process. I must go to a browser, put in a keyword phrase, see what number of pages show up in Google, input that number into a spreadsheet and move on the next one.
To do it under script control all that is done automatically for me. I don't have to do a thing other than feed my script the list of keywords. It will interrogate Google automatically, get whatever metrics I need from Google, and put it all nicely into a form that can be imported into my spreadsheet in one fell swoop.
What takes me hours to do manually only takes about half an hour to do under script control. And that is only because I can set the script to access Google in random fashion like all such software does.
Carlos
Quote: Originally Posted by Cricket Do you need more than what Google offers within their Webmaster Tools in terms of tracking your placement in the SERPS? And as far as keyword research, have you not found Google's AdWords Keyword Tool to be effective?
This might be the happy medium for you to stay within TOS while still saving some time getting the information that you want. Quote: Originally Posted by robjones I'm on the same side as you when it comes to following TOS. Those that show flagrant disregard for them tend to be shooting stars... they look good for a short while and flame out when they get caught and banned. Hi Rob. I agree that abiding by TOS rules is the way to go in general but I disagree that those who violate such rules with flagrant disregard end up being able to do so only for a short time. If one does it smartly one can get away with TOS violations all over the place.
I guess the reason I say that is that I have been reading a whole lot of black hat SEO stuff lately from of the masters in that field of SEO and sad to say...on the internet as far as Google ranking is concerned, such techniques pay off handsomely. I don't believe, by the way, that black hat is necessarily evil just because Google does not like it though certain aspects of it are just wrong (i.e. throwing all ethics and moral considerations to the wind in the pursuit of making money).
Quote: That said, on the topic of checking your own PR I honestly dont think using software to check it on occasion is what G was trying to avoid when they wrote that rule. Could be but how do we know that Rob? We don't. Google would love it if no one used ANY automated means at all to check their PR or ranking position. But that's unrealistic with respect to what they can and do enforce.
The rule is the rule and most of us are breaking it whether we realize it or not whever we use a PR checking toolbar or otherwise. Google does not make the PR available through their API or other authorized means.
Incidentally I wasn't referring to PR but that too is a violation of their TOS. I mean using automated means to determine PR such as using a Firefox plugin to get the PR of all sites.
Quote: Sometimes rules are written simplistically because brevity aids in the probability they might get read, and as a result something that isnt meant to be verbotten appears so. Admittedly so Rob but where does leave someone like me who reads the rule to determine what I can and cannot do? Do I stick to the letter of such rules when they were never meant to be read literally? Do I assume they were written simplistically and just violate them at will?
Quote: That's just my opinion of course, and I'd love to see someone ask google that question in their webmaster section and see what they say. The problem with asking Google is two fold. They are not going to okay a violation of their literal rule publicly. So the rule will remain the rule.
While all along their rule will either be unenforceable or selectively enforced by Google itself.
Which brings us right back to whether or not we should abide by the literal reading of said rules or not and leaves us in the same ethical/moral dilemna.
Do we or do we not abide by a literal reading of said TOS rules? \
If we abide by a literal reading we disadvantage ourselves. If we bend the rules to suit our purposes where do we draw the line in ignoring the TOS?
Carlos Something at Google to consider
http://www.google.com/support/websea...n&answer=86640
This is an old article, but might be something to consider.
http://news.cnet.com/2100-1023-883558.html
Another old one, but might be worth the read.
http://www.webmasterworld.com/google/3716136.htm
and . . .
Quote: Google has made efforts to detect and deny automated searches, which will limit the number of queries one can run. You may also run the risk of being recognized as an automated tool, and could potentially have your IP address shut down. http://www.searchenginejournal.com/h...-scanner/6420/ As an additional insight into what I am asking.
I can access Google with a browser called Lynx. A browser that even Google recommends in their help to allow one's to see how their crawler sees a page.
That browser just happens to have the capability of saving each page accessed automatically.
I can write a script to command Lynx to go and get a page of SERP results at Google. And my script can then process each page in whatever way I want.
I am accessing Google through a browser. No violation there.
But I am taking what that browser returns and doing whatever with it. Violation? Uncertain.
Regarding even the concept of automation...even that is confusing.
If I enter a keyword phrase in Firefox it formats that phrase in a way that Google understands and executes a search query for me. Is that automated? You bet it is yet I don't think Google has a problem with that. Why not?
Furthermore I can enter HTTP protocal commands into a command line and access Google that way too. Is that considered automated? I doubt it.
But if that isn't considered automated then what does a browser do if not use internal programming commands to do for us what we would have to otherwise do manually from a command line.
Carlos It seems to me that they are pretty specific?
Quote: Google's Terms of Service do not allow the sending of automated queries of any sort to our system without express permission in advance from Google. Sending automated queries absorbs resources and
I am hoping that this will be a discussion and not just the usual on this forum where I post a couple of questions...a few respond with very short snippets of input...and that's that for the thread (that does not always happen mind you but it does seem to happen fairly often which is why I generally hang out at other forums).
Generally speaking I believe in abiding by TOS rules and I do not encourage breaking them. Mainly from an ethical even a moral standpoint with respect to doing what is right by both God and man.
But...something I have been thinking of lately is that this places me in a disadvantaged position while pursuing internet marketing success.
For example...
Google's TOS says explicitly that no automated means can be used to access their service. What this means according to the common definition of what "automated" means is that one cannot use ANY software that automatically interrogates Google for ranking positions.
If one is going to keep track of where one's site ranks for a given phrase at any given time the only option available to a person who wants to keep track of their ranking is to do it "manually" by doing a manual search every time they want their rank.
While everyone else violates the Google TOS left and right using such popular software as Niche something or other, Samaria whatever, and any number of such software I am stuck having to manually look up my site rankings every single time (and that's not the half of it...I also look up Google number manually when doing keyword research).
That's time consuming and if time is money I am at a distinct disadvantage when it comes to making efficient use of my time. All because I want to abide by a TOS.
Now don't get me wrong. Time disadvantage or not...abiding by agreements I make is an absolute principle I live by and advocate.
The problem is that Google itself does not enforce their TOS in such a way that it levels the playing field between those who would violate it at will and those who would not.
The other day I was riding a public bus and was presented with a similar dilemna. The rule on the bus said "No Eating or Drinking". Clear. No question about what it means.
I had a banana I wanted to eat. So I went to the bus driver and asked if I could eat a banana and looking at me somewhat surprised as to why I was asking her...she said that would be fine.
I sat down and ate my banana at which time it dawned on me that this was very much like the Google TOS situation.
There is a rule. No automated access to Google. Yet everyone who uses the aforementioned software does it.
So where does that leave me?
Do I not eat my banana because the rule says not to? Or do I just eat my banana with an awareness that the bus company does not enforce it's own rule and that everyone else who is hungry while riding a bus...well...eats their bananas without a second's hesitation?
I can write scripts to automate access to Google and save me a huge amount of time. Such access is not at all denied me. I can do what others are doing using their own software.
Should I do so? Should I violate TOS rules, the letter of such rules, if the rule maker itself is not enforcing them or can enforce them?
Where does one draw the line on such TOS violations? Do we throw all TOS obedience to the wind and just do whatever we want or can get away with in the realm of internet marketing?
Your thoughts?
Carlos Do you need more than what Google offers within their Webmaster Tools in terms of tracking your placement in the SERPS? And as far as keyword research, have you not found Google's AdWords Keyword Tool to be effective?
This might be the happy medium for you to stay within TOS while still saving some time getting the information that you want. I'm on the same side as you when it comes to following TOS. Those that show flagrant disregard for them tend to be shooting stars... they look good for a short while and flame out when they get caught and banned.
That said, on the topic of checking your own PR I honestly dont think using software to check it on occasion is what G was trying to avoid when they wrote that rule. Sometimes rules are written simplistically because brevity aids in the probability they might get read, and as a result something that isnt meant to be verbotten appears so. My suspicion is that such a rule is intended to disallow bots from running ongoing PR checks for the purpose of populating fields in websites. What you're wanting to do IMO meets the spirit of the rule though by a strict reading appears to be a breach.
That's just my opinion of course, and I'd love to see someone ask google that question in their webmaster section and see what they say. Thanks for suggestions Cricket but I think I probably didn't explain what I meant well enough.
Let's say that I am targeting a given phrase...let's see...how about "black tuxedos". I create a site page about black tuxedos. I want to see how well my SEO efforts are doing by periodically seeing where that page is ranking in Google for a search on "black tuxedos".
When I first create the page and Google detects it I rank at 123. I make a few changes on-page SEO wise and after the next Google crawl I move up to 87. I make some more changes and then move up to 53, then 15, and so forth. Or I might move down if my changes were not good.
So I am tracking my Google position over time.
Now I know that some will say that such tracking is useless and so forth but without getting into the relative merits or not of such tracking that is what I am doing. It is the best way that I know how to determine if my SEO efforts are meeting with success or not.
Now I am targeting a multitude of phrases with various keyword phrases at various domains.
To keep track of my positions for each phrase that I target is getting to be quite a bear.
That's what I meant cricket.
Respecting what Google offers through their Webmaster Tools I do not believe that such gives me the ranking positions that I am talking about. I could be wrong. I'll have to check into that but I doubt they provide that.
With regard to Google's keyword tool yes, that is what I use. But...there is NO way that I can do manually what I can do under automated script control to get the numbers from Google that allow me to pick out a very few possibly good keyword phrases from among hundreds.
To do it manually is a very time consuming process. I must go to a browser, put in a keyword phrase, see what number of pages show up in Google, input that number into a spreadsheet and move on the next one.
To do it under script control all that is done automatically for me. I don't have to do a thing other than feed my script the list of keywords. It will interrogate Google automatically, get whatever metrics I need from Google, and put it all nicely into a form that can be imported into my spreadsheet in one fell swoop.
What takes me hours to do manually only takes about half an hour to do under script control. And that is only because I can set the script to access Google in random fashion like all such software does.
Carlos
Quote: Originally Posted by Cricket Do you need more than what Google offers within their Webmaster Tools in terms of tracking your placement in the SERPS? And as far as keyword research, have you not found Google's AdWords Keyword Tool to be effective?
This might be the happy medium for you to stay within TOS while still saving some time getting the information that you want. Quote: Originally Posted by robjones I'm on the same side as you when it comes to following TOS. Those that show flagrant disregard for them tend to be shooting stars... they look good for a short while and flame out when they get caught and banned. Hi Rob. I agree that abiding by TOS rules is the way to go in general but I disagree that those who violate such rules with flagrant disregard end up being able to do so only for a short time. If one does it smartly one can get away with TOS violations all over the place.
I guess the reason I say that is that I have been reading a whole lot of black hat SEO stuff lately from of the masters in that field of SEO and sad to say...on the internet as far as Google ranking is concerned, such techniques pay off handsomely. I don't believe, by the way, that black hat is necessarily evil just because Google does not like it though certain aspects of it are just wrong (i.e. throwing all ethics and moral considerations to the wind in the pursuit of making money).
Quote: That said, on the topic of checking your own PR I honestly dont think using software to check it on occasion is what G was trying to avoid when they wrote that rule. Could be but how do we know that Rob? We don't. Google would love it if no one used ANY automated means at all to check their PR or ranking position. But that's unrealistic with respect to what they can and do enforce.
The rule is the rule and most of us are breaking it whether we realize it or not whever we use a PR checking toolbar or otherwise. Google does not make the PR available through their API or other authorized means.
Incidentally I wasn't referring to PR but that too is a violation of their TOS. I mean using automated means to determine PR such as using a Firefox plugin to get the PR of all sites.
Quote: Sometimes rules are written simplistically because brevity aids in the probability they might get read, and as a result something that isnt meant to be verbotten appears so. Admittedly so Rob but where does leave someone like me who reads the rule to determine what I can and cannot do? Do I stick to the letter of such rules when they were never meant to be read literally? Do I assume they were written simplistically and just violate them at will?
Quote: That's just my opinion of course, and I'd love to see someone ask google that question in their webmaster section and see what they say. The problem with asking Google is two fold. They are not going to okay a violation of their literal rule publicly. So the rule will remain the rule.
While all along their rule will either be unenforceable or selectively enforced by Google itself.
Which brings us right back to whether or not we should abide by the literal reading of said rules or not and leaves us in the same ethical/moral dilemna.
Do we or do we not abide by a literal reading of said TOS rules? \
If we abide by a literal reading we disadvantage ourselves. If we bend the rules to suit our purposes where do we draw the line in ignoring the TOS?
Carlos Something at Google to consider
http://www.google.com/support/websea...n&answer=86640
This is an old article, but might be something to consider.
http://news.cnet.com/2100-1023-883558.html
Another old one, but might be worth the read.
http://www.webmasterworld.com/google/3716136.htm
and . . .
Quote: Google has made efforts to detect and deny automated searches, which will limit the number of queries one can run. You may also run the risk of being recognized as an automated tool, and could potentially have your IP address shut down. http://www.searchenginejournal.com/h...-scanner/6420/ As an additional insight into what I am asking.
I can access Google with a browser called Lynx. A browser that even Google recommends in their help to allow one's to see how their crawler sees a page.
That browser just happens to have the capability of saving each page accessed automatically.
I can write a script to command Lynx to go and get a page of SERP results at Google. And my script can then process each page in whatever way I want.
I am accessing Google through a browser. No violation there.
But I am taking what that browser returns and doing whatever with it. Violation? Uncertain.
Regarding even the concept of automation...even that is confusing.
If I enter a keyword phrase in Firefox it formats that phrase in a way that Google understands and executes a search query for me. Is that automated? You bet it is yet I don't think Google has a problem with that. Why not?
Furthermore I can enter HTTP protocal commands into a command line and access Google that way too. Is that considered automated? I doubt it.
But if that isn't considered automated then what does a browser do if not use internal programming commands to do for us what we would have to otherwise do manually from a command line.
Carlos It seems to me that they are pretty specific?
Quote: Google's Terms of Service do not allow the sending of automated queries of any sort to our system without express permission in advance from Google. Sending automated queries absorbs resources and