Using Client Certificates with screen-scraper

Posted in Miscellaneous on 01/21/11by Todd Wilson

We recently added experimental support to screen-scraper for client/PKI certificates.  Some web sites require that you supply a client certificate, that you would have previously been given, in order to access them.  I say this new feature is “experimental” because we’ve only been able to perform limited testing with it.  So far, it does seem to be working as it should, though.

In order to account for sites that use client/PKI certificates, we’ve added a feature to screen-scraper that allows it to use JKS files. These are files used by Java that encapsulate secure certificates and such.  The trick is to turn your existing client certificate file(s) into a .jks file.  We’ve currently only tested the feature using .pfx files, which we converted into a .jks file via the method described here:

http://www.cb1inc.com/2007/04/30/converting-pfx-certificates-to-java-keystores/

In the current alpha version of screen-scraper, if you look under the “Advanced” tab for a scraping session you’ll see a box where you can enter the location of your .jks file, and a box that will take the password you used when generating the .jks file.  There are also corresponding boxes under the “Advanced” tab for a proxy session.

If you’d like to use this new feature you’ll likely need to do some of your own research on how to turn your client certificate file(s) into a .jks file.  Here are a few sites that may help you in this:

You might also find these tools to be helpful:

Because this is a particularly difficult feature to test, we’d appreciate any feedback from those who are able to try it.  If you get the chance to give it a whirl, please also take a minute to post to our forum or send us a message.

1 Comment »

  1. Sonu said,

    November 6, 2014 at 11:37 pm

    Hello, neat idea! One suggestion toghuh; we’ve open-sourced an *almost* complete implementation of linux terminal in pure Python a couple of month ago. Seems like a perfect match for `ashttp` . Any feedback is much appreciated 🙂

Leave a Comment