Automated Screenshots

March 21, 2022

How To Detect Annoying Website Overlays, Modals and Popups

It’s a wild, wild, web out there. There’s all sorts of odd servers, doing dodgy stuff when you visit them and take a screenshot.

Alan Donohoe
Alan Donohoe
5 min read
Share this article:

In London, if a film, joke, your new jacket, etc, gets approval from your friends, you’ll hear: “Quality, mate. Quality”

You want your users to be saying that same phrase when they see a quality screenshot in your app.

You’ve invested months creating a quality user experience for your customers. The last thing you want is for it to be ruined by a lousy screenshot.

It’s a wild, wild, web out there. There’s all sorts of odd servers, doing dodgy stuff when you visit them and take a screenshot. You’ve got cookie banners, modal overlays, paywalls, or just plain nasty broken pages and servers. It’s impossible to know what surprise awaits when you visit various sites and take a screenshot.

So how do you handle all the oddness the world wild web throws at you? There are a few ways…

1. Manually replacing bad screenshots

“This screenshot is broken” is an all too familiar customer support subject line.

If you’ve maintained an app that includes screenshots and has users it’s only a matter of time before they start coming in. You’ll learn a lot about all the outliers and oddities out there. But often the fastest thing to do is manually take a screenshot, make some adjustments in photoshop and upload it. It’s not so bad doing this a few times a week.

As usage grows it’ll become increasingly frustrating for you and your users.

2. Writing fixes for edge cases

Few developers will be happy with a manual solution.

You might consider every broken screenshot as a test case for a bug to be fixed. You’ll amend your screenshotting code to handle every edge case you find. From then on, those issues should not happen again.

It’ll be a process of continuous improvement:

Learning, fixing, screenshotting…

Learning, fixing, screenshotting…

This means your code responsible for taking screenshots, contains within it a history of fixes to deal with the odd websites that users have raised and you have fixed. Having been doing this for 9 years now, taking hundreds of millions of screenshots, we still come across new weird and wonderful render issues every single week. We find this fascinating and relish solving each one. But, understandably, that’s not everyone’s cup of tea.

3. Fixes For Specific Sites.

Some problems are specific to certain websites.

Your users might collectively have a few dozen that they’re most likely to screenshot. Making a recipe for each of them is often worth the investment. Tweaking the code taking the screenshot, such as adding a delay, clicking a certain element, hiding a certain element, etc.

You can find a full list of options that resolve 99% of problems you’re likely to encounter here: screenshot options.

So, if the site you are screenshotting has an annoying banner blocking the good stuff, you can simply use the hide_selector option and it will be gone…

4. Artificial Intelligence / Deep Learning.

As a team who likes to keep up to date with the most recent advances in technology, at Urlbox we have recently been looking at using AI / Deep Learning to automatically tell us, or our users, when there’s an issue with a screenshot.

Built using the Python Keras library, we have built, trained and deployed a Deep Learning Neural Network to automatically recognise successful or obscured screenshots. Like so...

Successful

Obscured

Using hundreds of screenshots created by Urlbox of various websites, and then manually classified into different categories to train the model, the model initially achieved accuracy rates of 95% on unseen data. With some improvements to how we process the data and a further round of cleaning the data so remove the more ambiguous images, we finally managed to create a model that achieved an accuracy rate on (unseen data) of 98.75%.

Which is very, very good.

If you’re interested in learning how you can train and deploy a Deep Learning Neural Network, we have a more technical deep dive coming soon.

We are considering extending the number of classifications, to include broken pages (500s, etc), paywall sites, and a model trained specifically to detect successful/ broken social media site screenshots.

And of course, we are exploring how this can help our users...

Want an end of day email with a summary of your successful vs unsuccessful screenshots from that day? Want an email alert when a site you are trying to screenshot seems to be having issues?

Conclusion

We would love to hear any ideas you might have about how you might find this useful. We have a ton of ideas on this, but we like learning about our users’ worlds, so we can make sure we are helping them in the best way possible.

So these are some of the things we do, constantly, quietly, behind the scenes, for our users to make sure their screenshots are Quality, mate. Quality.

If you'd like to see the results of our ongoing quest for quality website screenshots sign up for a free trial here.

Free Trial

Ready to start rendering?

Designers, law firms and infrastructure engineers trust Urlbox to accurately and securely convert HTML to images at scale. Experience it for yourself.

7 day free trial.No credit card required.