Comments for screen-scrapeable http://blog.screen-scraper.com Thoughts, tips, and updates on screen-scraping Sun, 14 Mar 2010 14:27:53 +0000 http://wordpress.org/?v=2.5.1 Comment on Screen-Scraper Annotations for Java by Todd Wilson http://blog.screen-scraper.com/2009/11/11/screen-scraper-annotations-for-java/#comment-111429 Todd Wilson Thu, 17 Dec 2009 22:06:19 +0000 http://blog.screen-scraper.com/?p=88#comment-111429 Hi, Seems like we ran across an announcement on this one earlier, but I haven't yet seen it out in the wild. It's quite rare that we encounter a site that we can't scrape, though. Not knowing exactly how this utility works I can't say for certain just how we'd handle it, but at some point we may need to try :) Todd Hi,

Seems like we ran across an announcement on this one earlier, but I haven’t yet seen it out in the wild. It’s quite rare that we encounter a site that we can’t scrape, though. Not knowing exactly how this utility works I can’t say for certain just how we’d handle it, but at some point we may need to try :)

Todd

]]> Comment on Screen-Scraper Annotations for Java by jon rios http://blog.screen-scraper.com/2009/11/11/screen-scraper-annotations-for-java/#comment-111422 jon rios Thu, 17 Dec 2009 19:15:20 +0000 http://blog.screen-scraper.com/?p=88#comment-111422 like your dedictation to data extraction on the web, have used several products in the past and just found yours.. I am evaluating it for last 2 days... i wanted to bring to your attention this company that sells SW that does not allow "bots" to scrap pages, http://www.pramana.com i assume you aer aware of them or similiar companies.... just wanted to get your expert advice on their technology and what the potential impact on scrapping, if any, or even on your own product (i would assume none since it seems you work the extraction through a saved proxy session and dont scrap directly from the site..at least this is my initial understanding of your functionality over others, please conrrect me if i am wrong) thks and nice work like your dedictation to data extraction on the web, have used several products in the past and just found yours.. I am evaluating it for last 2 days…

i wanted to bring to your attention this company that sells SW that does not allow “bots” to scrap pages, http://www.pramana.com

i assume you aer aware of them or similiar companies…. just wanted to get your expert advice on their technology and what the potential impact on scrapping, if any, or even on your own product (i would assume none since it seems you work the extraction through a saved proxy session and dont scrap directly from the site..at least this is my initial understanding of your functionality over others, please conrrect me if i am wrong)

thks and nice work

]]> Comment on Screen-Scraping Ethics by webscraper http://blog.screen-scraper.com/2008/04/21/screening-scraping-ethics/#comment-84991 webscraper Wed, 04 Mar 2009 06:53:38 +0000 http://blog.screen-scraper.com/2008/04/21/screening-scraping-ethics/#comment-84991 thanks, Todd thanks, Todd

]]> Comment on Screen-Scraping Ethics by Todd Wilson http://blog.screen-scraper.com/2008/04/21/screening-scraping-ethics/#comment-84011 Todd Wilson Wed, 25 Feb 2009 22:53:52 +0000 http://blog.screen-scraper.com/2008/04/21/screening-scraping-ethics/#comment-84011 Hi, The easiest way would probably be to sprinkle calls to session.pause (http://community.screen-scraper.com/API/pause) throughout your scripts. Todd Hi,

The easiest way would probably be to sprinkle calls to session.pause (http://community.screen-scraper.com/API/pause) throughout your scripts.

Todd

]]> Comment on Screen-Scraping Ethics by WebScraper http://blog.screen-scraper.com/2008/04/21/screening-scraping-ethics/#comment-83993 WebScraper Wed, 25 Feb 2009 21:47:15 +0000 http://blog.screen-scraper.com/2008/04/21/screening-scraping-ethics/#comment-83993 good one. How can I do adjust screen scraper to run more slowly, so as to minimize the strain on the server? thanks, good one.
How can I do adjust screen scraper to run more slowly, so as to minimize the strain on the server?

thanks,

]]> Comment on On the Cusp of a Public Release by Todd Wilson http://blog.screen-scraper.com/2009/02/02/on-the-cusp-of-a-public-release/#comment-83992 Todd Wilson Wed, 25 Feb 2009 21:41:46 +0000 http://blog.screen-scraper.com/?p=81#comment-83992 Hi, We actually removed that option in version 4.0. The recommended approach now is to run a scraping session directly, then utilize an initialization script that gets run before the scraping session begins to set up variables and such. If you have other questions on this would you mind posting to our forum (http://community.screen-scraper.com/forum)? That way others can benefit from the answers as well. Thanks much, Todd Hi,

We actually removed that option in version 4.0. The recommended approach now is to run a scraping session directly, then utilize an initialization script that gets run before the scraping session begins to set up variables and such.

If you have other questions on this would you mind posting to our forum (http://community.screen-scraper.com/forum)? That way others can benefit from the answers as well.

Thanks much,

Todd

]]> Comment on On the Cusp of a Public Release by Michael Shiels http://blog.screen-scraper.com/2009/02/02/on-the-cusp-of-a-public-release/#comment-83967 Michael Shiels Wed, 25 Feb 2009 18:56:44 +0000 http://blog.screen-scraper.com/?p=81#comment-83967 What happened to the 'run script' right click menu option? I updated since you are indicating it's pretty solid, but wow, I need the ability to run scripts at least, that's something I can't seem to find anywhere. What happened to the ‘run script’ right click menu option? I updated since you are indicating it’s pretty solid, but wow, I need the ability to run scripts at least, that’s something I can’t seem to find anywhere.

]]> Comment on On the Cusp of a Public Release by Todd Wilson http://blog.screen-scraper.com/2009/02/02/on-the-cusp-of-a-public-release/#comment-78790 Todd Wilson Wed, 04 Feb 2009 13:35:33 +0000 http://blog.screen-scraper.com/?p=81#comment-78790 We do release notes for each major version here: http://www.screen-scraper.com/release_notes/screen-scraper_release_notes.php#pr4 We actually have an item in our to-do list to include release notes with each alpha in the screen-scraper workbench as well. That way you know what you're getting each time you upgrade. Watch for that in a future release. We do release notes for each major version here:

http://www.screen-scraper.com/release_notes/screen-scraper_release_notes.php#pr4

We actually have an item in our to-do list to include release notes with each alpha in the screen-scraper workbench as well. That way you know what you’re getting each time you upgrade. Watch for that in a future release.

]]> Comment on On the Cusp of a Public Release by Chris http://blog.screen-scraper.com/2009/02/02/on-the-cusp-of-a-public-release/#comment-78627 Chris Wed, 04 Feb 2009 03:36:24 +0000 http://blog.screen-scraper.com/?p=81#comment-78627 Nice! Those methods will definitely be useful. How do you guys do release notes, by the way? Just for public releases, like here in this blog, or are there more official notes for minor updates? Nice! Those methods will definitely be useful. How do you guys do release notes, by the way? Just for public releases, like here in this blog, or are there more official notes for minor updates?

]]> Comment on On the Cusp of a Public Release by Todd Wilson http://blog.screen-scraper.com/2009/02/02/on-the-cusp-of-a-public-release/#comment-78190 Todd Wilson Mon, 02 Feb 2009 21:22:14 +0000 http://blog.screen-scraper.com/?p=81#comment-78190 Hi Chris, Thanks for the feedback. On your first point, these methods are available (just not publicly documented yet since it's still an alpha feature): session.logDebug session.logInfo session.logWarn session.logError On your second point, the best approach would probably be to enhance the data set viewer. I'll make sure we have something like that in our to-do list. Outputting it to the log was actually an unintended side effect of an internal change we made. Hi Chris,

Thanks for the feedback. On your first point, these methods are available (just not publicly documented yet since it’s still an alpha feature):

session.logDebug
session.logInfo
session.logWarn
session.logError

On your second point, the best approach would probably be to enhance the data set viewer. I’ll make sure we have something like that in our to-do list. Outputting it to the log was actually an unintended side effect of an internal change we made.

]]>