Version 7.0.1a released

Posted in Updates on 04.19.16 by jason

When you updated to version 7.0.1a, the first thing you’ll notice is spruced up GUI, but there is a quite a bit going on under the hood too. You can see all the release notes here.

If you want to use this update, here is the instruction to update.

Screen-scraper 7.0 Released

Posted in Updates on 03.02.16 by jason

This new stable version adds many new features, and give you the ability to scrape sites that are using the lastest SSL features.

The installers are available on the download page.

Screen-scraper 7.0 requires a newer JRE than the previous stable release, therefore upgrading requires some additional steps.

If you don’t already have all your scrapes exported, or just want to preserve the current configuration, you need to upgrade your current screen-scraper to the latest alpha version 6.0.69a (Instructions). Once done, back up the content of the screen-scraper/resource/db directory.

Linux/OSX

  1. The new installer does not include the JRE
  2. You need to have the Java JRE 1.8 installed (1.7 will work, but is not recommended)
    1. Make note of the install location (a symlink isn’t valid)
  3. Run the new setup SH file.
    1. You cannot install over the top of an exiting installation. You must either move the current directory or during installation choose a new install location.
  4. In the screen-scraper install directory, locate and edit both the server and screen-scraper script. On the line “INSTALL4J_JAVA_HOME_OVERRIDE” (at the top), add the path to your JRE install

Once done, you can replace the content of the resource/db directory with the one you’d backed up.

Windows

  1. Make sure screen-scraper is not running (neither the application, nor in sever mode)
  2. Run the setup EXE
    1. You cannot install over the top of an exiting installation. You must either move the current directory or during installation choose a new install location.

Once done, you can replace the content of the resource/db directory with the one you’d backed up.

We recommend this update for all scrapers, and if there is any problems, please let us know here or on the support forum.

 

HTTPS connection issues

Posted in Updates on 04.29.15 by jason

We’ve been seeing lots of issues with scrapes connecting to HTTPS sites. Some of the errors include

  • ssl_error_rx_record_too_long
  • An input/output error occurred while connecting to https:// … The message was peer not authenticated.
  • javax.net.ssl.SSLPeerUnverifiedException: peer not authenticated

The issue came about when the Heartbleed vulnerability necessitated changes to some HTTPS connections—some of types aren’t secure anymore, and new versions have come out. Screen-scraper needed two changes to catch up, and they are:

  • Update to use Java 8
  • Update of HTTPClient to 4.4

Both of these are pretty large changes, so they aren’t in the stable release yet, however in some cases they are the only option to make a scrape work, therefore here is the instructions to get what you need. Read the rest of this entry »

Version 6.0.18a of screen-scraper Released

Posted in Updates on 10.16.12 by Todd Wilson

A few minor updates in the one, along with a long-awaited global find feature!

Version 6.0.14a of screen-scraper Released

Posted in Updates on 06.28.12 by Todd Wilson

Several small changes in this one:

  • Extractor patterns invoked manually can now be tested on a sub-set of the HTML page.
  • Added scrapeableFile.setForcePOST.
  • Upgraded internal GWT libraries.
  • Prettied up the web UI.

Check the alpha log for a full list of changes.

New Quick Guide video

Posted in Tips, Updates on 06.15.12 by scottw

We recently released a new Quick Guide video.  In less than three minutes you can get an idea of what it’s like to use screen-scraper.

Source

Version 6.0.6a of screen-scraper Released

Posted in Updates on 05.10.12 by Todd Wilson

Several small changes in this one:

  • Upgraded Bean Shell to the latest version.
  • Searches within a proxy session now include notes.
  • Fixed an issue that would cause the workbench to freeze when the breakpiont window was up.
  • Now using global proxy settings if no session proxy settings are found.
  • Improved cookie handling in the proxy server.
  • Fixed a bug that would cause a proxy session to not be completely saved.
  • Added sutil.makeGETRequestNoSessionProxy.

Version 6.0.4a of screen-scraper Released

Posted in Updates on 05.01.12 by Todd Wilson

Several bug fixes and new features in this release:

  • Fixed a bug such that a scrapeable session ID is now being generated even for scraping sessions that will run in the future.
  • Fixed a bug where nodes in the tree weren’t being highlighted correctly.
  • Scrapeable files can now be added via a URL.
  • If the DatabasePort and WebServerShutdownPort properties are omitted from the screen-scraper.properties file they’ll now be automatically set to the value of an open port.
  • The ProxyPort will now only be tested and used when screen-scraper is running in server mode if the AllowProxyScripting is set to true.
  • Added a “Load Response from Clipboard” button to the scrapeable file panel.
  • Updated BeanShell to the latest version, disabling unstable Windows scripting in the process (e.g., VBScript).

Version 6.0 of screen-scraper released!

Posted in Updates on 04.11.12 by Todd Wilson

It’s now official!  Precisely one year after our last major release (5.5) we’ve let loose version 6.0.  This is undoubtedly our most feature-rich rock-solid version yet.  Take a glance through the release notes for a list of the myriad of changes.  Better yet, either upgrade your existing instance, or download it fresh!

Version 5.5.43a of screen-scraper Released

Posted in Updates on 03.27.12 by Todd Wilson

Just a couple of small changes:

  • Updated session.downloadFile to use current proxy settings.
  • Updated tool-tips in script editor to accurately reflect API changes.
With a little luck we’ll be having a 6.0 release in the next week or so…

Previous Entries »