GTAC 2009 – Fighting Layout Bugs

>> All right, so we move on to the second regular talk. We’ll move away from specification but to a nevertheless incredibly challenging problem that of layout bugs. Michael is going to present it there >> TAMM: Yeah. Welcome to my talk, “Fighting layout bugs.” First slide “Who am I?” I’m a system architect at Optivo. I’m an active WebDriver committer and regular conference speaker. I haven’t put it in a book yet but put several JUnit tests [INDISTINCT] book Okay, maybe not such a good comparison. But there are also some tools I like to use. Big fan of IntelliJ IDEA, Hamcrest, Mockito, FindBugs just to get a little picture of me. So, here’s the agenda. I would like to present to you suite techniques to fighting layout bugs I guess the most interesting test technique and the main part of the talk is the last one, detecting typical layout bugs using image processing and JavaScript. But first things first, first technique named “Continuous HTML validation.” The idea is to prevent layout bugs by always delivering valid HTML pages That idea is effacing but new. But I’m curious who has integrated HTML validation and it’s a direct into a basis and its development process. For people is a sort of a little bit more. Okay, so how do we do it? The first thing that came to my mind is maybe you already know there are some browser extensions which do have HTML validation. But those only work in one browser and they don’t validate pages automatically. You always have to click some buttons to trigger the validation. Even might argue as well. That’s okay. Most [INDISTINCT] only–mainly work as one browser and cannot do this. But there’s one killer argument against browser extensions is that browser extension can’t make the build fail that says invalid HTML. And that’s what we want, don’t we? So, another idea might be to scan files during the build process. So, that you get something like compiler errors but it turns out it isn’t a good idea for HTML validation because most of the HTML is some kind of just P-files, text files, some kind of templates or even on Java code. And so it’s doesn’t–[INDISTINCT] HTML files anymore but there’s a lot of presentation [INDISTINCT] it so. Well, I guess it’s very hard problem to scan for valid HTML files as templates, so this is not a good idea So the–maybe the good idea is to validate HTML during your front end tests, yeah, as oppose you have some kind of continuous indication server and they run your TestSuite. I hope some of you guys already have front end test which that runs on the continuous indication server. So why not start with a very simple separate test and visit some well-known pages, start with your home page and then I guess you know which pages–especially on your website that you need some attention and just preps the HTML and send it to valid data and see if you have valid HTML. I mean why is it good to have valid HTML by the way? If you don’t have valid HTML, the browser must guess about what you really wanted to have for a dump

feed. And if it guesses wrong or if you just forget a closing tag, the dump feed is not what you expect when you find your CSS. And so your CSS selectors don’t match anymore, and then you will get layout bugs. Another option would be if you have this code which validates HTML, why have a separate front end test, if you already have a test feed of several front end test then just check every time you get a page, if it’s valid HTML And of course, you shouldn’t reinvent the [INDISTINCT]. They all are already tools which validate HTML [INDISTINCT] or maybe use W3C Markup Validation Service. This is a service that you can download and install and it is also a publicly available server from the W3C. You can validate by putting in URL to your website or by uploading an HTML file or by direct input. And, you know, we have some sample HTML with a missing closing tag And here are some options. And if you–let’s check, you get the select bar and also the small red icon, that icon. And if you scroll down, you get–for each warning or error which is detected to get a good description why is this wrong. So, it’s quite useful and it’s maintained here. It’s always improved. And it comes from W3C. So I guess we should be using this for validation. But [INDISTINCT] this approach, because I want it to fail faster than the continuous indication, both machine detects such as invalid HTML. I want it to whether it’s detected on the developer machine, when you use developing HTML or templates or whatever. So, I came up with another solution and it was a simpler servlet filter. How does it work? The first–plus point is it works in all browsers, of course, because it’s in the server site. And it validates HTML automatically, yeah cool. So, but some other requirements I had is should not hamper daily work. Developers should have benefit from it, but it should not separate from it by slower loading transit of pages. And of course, it should also use W3C Markup Validation Service. Yeah and I hoped such class [INDISTINCT] and here is

it how it works. I hope you are a little bit familiar with the servlet spec. It has a doFilter method in each servlet filter. And in there, I’ve wrapped the http response in the “TeeHttpServletResponse” And then I say “doFilter” and the “filterChain” sort of request this process and for the “request” and this “responseWrapper”. That’s two things First, it immediately writes back all the bytes to the browser. So, the HTML goes into the browser as fast as always. And the same thing is, that’s why it’s called “TeeHttpServletResponse”, it–at first, the response then an internal byte away. And when this message returns, the doFilter message, I have the whole response in the browser. And the browser can do is [INDISTINCT] stuff as well. I also have my buffer. And if it’s really isn’t HTML which was served, it might could be something else, CSS or an image, whatever, but if it’s HTML, it’d fetch the HTML from the buffer and send it to the Markup Validation Service and then it get an answer that to agree valid or not And then I haven’t closed the response yet, I haven’t closed the output stream. But what I do know is after the closing HTML check, I append this small JavaScript, snippet it which displays a little box which is either green or red. So each developer will be able to see if it’s HTML is valid or not. Of course I’ve sort of a demo for this. Let’s see if this works. This is a small Web App where I downloaded some Web pages from that and this has description. And you see, here’s the servlet filter. It checks all works and it currently uses a–publicly, I’ll be able to have valid data. But of course if you do this internally in your company, you should install your own validation service. So, you have it on the internet. Yeah. Let’s start it from scratch. And here we go. So what do you see now is the HTML page. And you see this yellow box there and now, it turns red, wow, great. Well, not for eBay, maybe. And I can click on the view with that button and then I get all the errors which are there

Okay, I don’t want get too [INDISTINCT] right here. And I also wanted to show you a valid HTML page across [INDISTINCT]. There are some on the internet. Okay, it works fine. Some final thoughts for this to [INDISTINCT], “Try it out”. The servlet filter is an open source Just download it, try it out, and [INDISTINCT] appreciate it, of course. Another point is how to deal with ads. Sometimes if you have an enterprise website, you need to integrate third party HTML snippets, ads or whatever And of course, sometimes those third parties snippets are invalid. Mm… Now, what to do it about it? You can’t change them because you have to integrate them. The first thing, maybe most interesting to do is to indicate the switch into your web application to turn off ads, you know. That’s nice. If you develop and don’t see the ads, you know, I think it’s cool. It makes your website faster. And if you can’t do this maybe it’s important to add those snippets, most easy thing to do is to have some special HTML commands in your produced HTML and extend the servlet filter And before you send the HTML to the validation service, just strip out those third party HTML snippets that you are not interested in validating. So you only check your own HTML which is surrounding those snippets Yes, so much for the technique number one Technique number two, a little bit similar, continuous CSS validation. Same idea, prevent layout bugs by always delivering valid CSS And once again, I would like to ask “Who does this?” [INDISTINCT] regular buzzes and its development process?” Wow two guys, cool Okay this is a [INDISTINCT] developer, I guess Once again, it’s so easy to have a typo in this CSS [INDISTINCT] and whole block of CSS will be ignored by the [INDISTINCT]. I mean CSS is quite robust, so, if a block is not valid, it just takes the next one. But yeah, I guess the block which is ignored has some meanings, so, again, you will probably have a layout bug somewhere, if you somehow enter some characters, of course, you wanted to comment it and it should be comment command but the comment [INDISTINCT] didn’t go up in time. I guess you all know this kind of errors. So, we definitely want to have valid

CSS. So, yes, they have browser extensions, of course, but this is the same slide that you already saw, same arguments applied. You can’t make the build fail. So can we maybe scan files during the build process? Mm CSS is not only a CSS file. It’s also an HTML-style elements or even style attributes but nevertheless why not validate CSS files? It turns out that most of the CSS normally is a CSS files and style elements or its attributes are–not so much used. I guess it depends from website-to-website But often, there are a lot of CSS files and, yeah, why not check them? So, of course, we don’t want to [INDISTINCT] and yeah, there is a CSS validation service already provided by the W3C. And speaking of layout bugs, there is one layout bug here, does anybody see it? It’s hard to spot. It didn’t occur to me for a long time. I guess it wasn’t. First I sought–oh, this is by intent, it’s either one that’s designed to be. So, I did some investigation There’s an image towards the upper right corner And even W3C don’t get it right. I guess there’s an issue about layout bugs. Should you use this validation service? It complains about something we know as CSS hacks, like this one. We don’t want to have warnings than we do such things by purpose, but only about some CSS hacks. There are a lot of CSS hacks which are absolutely okay. The style the CSS hack which often used for IE is absolutely okay. So you can go with CSS hacks and don’t get warnings. But then again, the CSS Validation Service complains about unknown CSS properties Most of the time [INDISTINCT] the specific ones, like, [INDISTINCT] or the typical ID ones that’s in lined with JavaScript expressions [INDISTINCT]. I don’t know. We did two years ago, and our project was to use this CSS Validation Service. The nice thing about this, it’s actually Java code, because you can download and it’s an open source so we can modify it and twist it your needs. The Markup Validation Service is a [INDISTINCT] bunch of scripts that need to install in a PC server. But this is just plain Java code and a bit of module around it. This is a snippet from our–from HTML And what we did was to validate all CSS files except those which are meant to be for the IE, which were included by this conditional command in the HTML. I guess you all know So, this doesn’t validate all CSS but a lot

of CSS. Yeah, it worked quite well. And I thought about open sourcing this module too, but [INDISTINCT] all the [INDISTINCT] of it Which basically is important, fits on one slide so I decided not to start another [INDISTINCT] that can put it on one slide. Yeah, I don’t want to go into details here, you can see–you can have a look at it later. If you want to [INDISTINCT] just download it and see if it fits somehow in your [INDISTINCT]. Of course, you could also validate CSS in your front end tests. I mean getting CSS files, you already know it’s a good idea. You should do it on your CE machine so it’s a build for fail But what about the CSS switches in HTML and Yeah, once again, use front end test. Why not? You can use this. It’s the same like the HTML validation [INDISTINCT] test. And why not have a JavaScript? I haven’t done that yet, it sounds like a nice idea to me Okay, that’s one problem I think. How do you get feedback, because you can’t attach JavaScript to CSS? You have to have some state of passage and [INDISTINCT] and maybe one will come up with a nice solution for this one day. But valid CSS is by far not good enough. This is a recording of a [INDISTINCT] public website from the German television station. And you see even if they had valid HTML and valid CSS, there are some [INDISTINCT] that cause a regress [INDISTINCT] found in CSS and gets flowing with four errors. Mm… So another thing is this is a special page in their website And I guess they had some design factoring and some CSS change. And there was one special page, that this is old HTML doesn’t fit to the new CSS. And so you get something totally potent, even if you have valid HTML and CSS Website doesn’t look good. And it just directly leads me to technique number three, detecting layout bugs using JavaScript and image processing What is the idea here? The idea is nothing less but detecting layout bugs by simulating the human eye. Wow, that sounds a little bit like artificial intelligence and–well, nobody has a definition for intelligence, so maybe it is. But it’s quite easier in the end, just some clever elements. So, let me show you

some more typical layout bugs. [INDISTINCT] The text goes on so it’s truncated here to a vertical edge and also here. Or another one, this is some widget, which there is not enough space for showing all these [INDISTINCT] text, so it’s placed in another corner in a row but you don’t see it. So, the text is truncated at the horizontal edge. Or this one, the text is not truncated here, but it overlaps the vertical edge and this doesn’t look professional. The best thing is if you have layout bugs, this just looks unprofessional and you don’t want people to think about your company to be unprofessional. And those layout bugs are everywhere, even on sites of big companies. Here, this is–most of the time, it’s a home page but some page deep down is a navigation hierarchy. This is my Yahoo profile This, I guess, is a localization issue. And German text is most of the time longer than English text, so it overlaps here [INDISTINCT] So, how can we–oh and another example. It has an overlapping but there is some space missing. This doesn’t look good together So, how to automatically detect those types of layout bugs? Here’s an idea and the first things is we need to know which pixels belong to text and then we need to know which pixels belong to a vertical or horizontal edge and if you know this, we can do some end. And if text pixels and edge pixels overlap, we have found a layout bug. And then of course, we need to somehow report those found bugs So, let’s start with detecting text. I put

in a simple TextDetector and it works like this, as a–its page has a front end test, [INDISTINCT] click for the page. And the first thing I do is I inject one of my favorite JavaScript library into the page, jQuery And then I do jQuery star, this selects all elements. And then I say CSS color black And this means all the text of every headline, every link, every paragraph becomes black So, I take a screenshot and then I say give all text on the page to color white. And then I take another screenshot. And all I have to do now is compare those two screenshots and all different pixels are probably text Now, let’s see how this works in practice This is a page I want to analyze. The first thing I do is make all the text black, there Then I do make all the texts white. And here’s the difference. Wow, really this is all the text pixels. The pop goes up in the station, it puts an ocean of page in the background [INDISTINCT] it out. There. Oh, it seems to work. Thus, problem solved, The next one, detecting vertical edges. I think edge detection is actually a problem already solved in computer graphics. There are sophisticated algorithms for it. But I don’t like sophisticated algorithms Most of the time, they are quite complex, and the only interesting in the vertical and horizontal edge [INDISTINCT] a little bit more easy algorithm. Once again, I work on some edge detector which injects jQuery, and to help myself, I saved all the text to transparent So, I have no text, so I can concentrate on the detecting edges. And of course, we’ll take a screenshot, and then I find vertical pixels sequences of a certain minimal lengths, where each pixel has a same color or at least a very similar one. And then I select those which have a high contrast to the left or the right, and these are my vertical edges, you know. I have to do some clean up. I only want vertical edges of certain lengths, but it looks quite well, we will see. Well, here is my page, this help us to let all the texts disappear. I’m not interested in text here, I’m interested in vertical edges. So, well, we have a lot of candidates here, these are all candidates for vertical edges; same colored pixel sequences. And if I look for pixel sequences which are high at only at high contrast to the left or the right., I get this–and last

step is you’re going have to fill up [INDISTINCT] which haven’t connect at minimal lengths, and it seems the minimal length here was 16 pixels. And oh, it seems like it detected all vertical edges. And, of course, it can do the same with horizontal edges. And if you put them together, you have all your vertical, horizontal edges, cool. So, we have our text pixels, we have our edge pixels, it’s dual and–hey, cool. That is, we found a layout bug, that’s important. I have a little class called layout bug, this is a markBuggyPixels What I do, if you’ll look back, this is how to see it. This is even the faded version of the website. So, what I’ll do is I replace each buggy pixel by a circle; its radius is 5 pixels, so it gets a buggy area. And then I find the outline of this buggy area and cause a thick red line around it. This is turns out to work really nice. So, here is my buggy area along the small circles or around some buggy pixels and this is the outline, an icon [INDISTINCT] in this. This is an automatically ejected layout work. The first time I saw this picture, A [INDISTINCT]. Wow, this really works. And of course, hazard types of layout bugs. Let’s see if this work; works right You see that there’s’ text there, I can’t read it, you know, can I know what what’s [INDISTINCT] there, but it’s hard to read And I think this as a layout bug; text, which is hard to read. There can be two reasons for it. Here, you can select different colors schemes but sometimes it’s also the case that your CSS is full or you have forgotten some CSS selectors or you have change to HTML and suddenly. You have select link texts on blue background or you have write link text on yellow background and, yeah, I see that’s quite often. So, how do you go about those types of layout bugs? Yeah, we need to know, which pixels belong to text; already solved this program. The next thing that we need to know is the background color around the text, that’s not hard to do, just get to the outline; also text pixel and look what color they have. And the last thing you need to know is the actual text color. It shouldn’t be that hard to do. You know, pink (ph) and black,

isn’t it? But if you zoom, this is what the actual–actually, blend (ph) out by the browser, the txt is initialized (ph) and if you look closely here to the arrow, this isn’t even black. This is brown, magenta blue; there’s no black pixel in there. But [INDISTINCT]–you don’t really have to know the exact color All you need to know is does it have the high contrast with the background. If not, it’s layout bug. So, I don’t want to get into the details here, because [INDISTINCT] exactly works but I want to announce (ph) an open source by the way. I have put online. It’s called fighting layout bugs. Try it out. What does it offer? Not much at the moment but it does [INDISTINCT] and DetectInvalidImagesUrls, early detector. This one scans for image elements which it [INDISTINCT] source attribute. Maybe the source attribute is empty or it points to an image, which produces a drawn (ph) at first points. Then we have this DetectTextNearOrOverlappingVerticalEdge and VerticalEdge detectors, which I just showed you. And we have this DetectTextWithTooLowContrast detector. Long [INDISTINCT], aren’t there (ph)? Yeah, and of course, I want to show you how it works. So, let’s start with this DetectTextOrOverlappingVerticalEdge test Let’s zoom a little bit. Here we go. I have [INDISTINCT] driver in the sense it’s a Firefox driver by the way. Currently only the Firefox driver is capable of taking a speed (ph) shot But the [INDISTINCT] be able to zoom and to open a driver, [INDISTINCT] which is–should be released, I hope this year, if not this year, then of course next year. And yeah, what do I do here? And just, in a sense it’s a detector class, then I say, [INDISTINCT] screenshot directory. I don’t have to do this but if I do, the detector will take a screenshot and not only see that this is a layout bug Then I say, find layout bugs and I give to [INDISTINCT]. This will analyze the current pitch and, yeah, hopefully it will turn to layout bug, which I print out here. So, let’s see if it works. It worked yesterday. You see the text detector – the text detector had work and, whoa, there it is, we have detected some text near [INDISTINCT]. And we have been warned, also [INDISTINCT] with those element And we have screenshot. Just take a look at it. Which one is it? It must be from today, this one. There it is. Detected text was smear or overlapping with glitch. Cool. Whoa. Thank you, thank you very much. Let’s play another one, DetectTextWithTooLowContrast. It’s actually the same code. Just another detector and another [INDISTINCT]. Here it is, detect text with too low contrast. Find layout bugs and let’s

see what happens. Yep, there it is. Detected text is too low contrast. Let’s take a look at the screenshot. There it is, DetectTextTooLowContrast for today. Yep, it worked. Once again. Okay It’s more on Outlook; of course, there are more typical layout bugs out there. And the most, I’ve seen one, I guess–I think it’s this one. You have different layout and different progress. This is Chrome and this is Firefox and you see there’s a button that’s behind the text have appeared on the next [INDISTINCT] But here’s the link, if not [INDISTINCT] or it’s, it’s on the next line. So normally, you don’t want this. Maybe it’s okay for you but normally you don’t want this. And how do you go about this? Then we have several [INDISTINCT] drivers, which then take screenshots We can use them and you need to somehow compare those screenshots. But we can’t of course do a pixel for pixel, because there might be slight difference as in internalizing (ph) So, I guess we need to come up with some posse (ph) algorithm to compare those screenshots And I haven’t tried it yet, of course, the content. [INDISTINCT] I can only take screenshots with Firefox at the moment but I think the first thing is I blow out the screenshot a little bit and then I shrink it to a certain size and then I can do a comparison with some FlashWords, so pixels don’t have to be as exact color but at least the same color and I think that should work. You will see then the next [INDISTINCT] final release comes out. Here’s another typical layout bug. This is lightbox which displays a video and unfortunately, the lightbox doesn’t have enough space to contain overview. So, you get scroll bars here. They don’t make much sense because, let’s say [INDISTINCT] the place would be [INDISTINCT]. So, it’s not nice, it looks unprofessional. What can you do about it? Yeah, scroll bars most of the time looks the same. But what we can do is a little bit like this Google screenshot actually, where find supplementary (ph) on your screenshot. Now, let’s detect scroll bars and images. And if you–you’re page doesn’t suppose to have a scroll bar, but you detect the scroll bar then we probably have a layout bug there The problem here, this is detect image is not common; there might be pages which have scroll bars by [INDISTINCT], which we have to go deal with this. But it should be sensible detector for pages that should not have scroll bars. Here’s another one I often see. There are some broken borders and you see some [INDISTINCT] then from here and here. There’s some space between he border and the box content. This one is a little bit harder but it still seen–this is automatically detectable because just ask yourself, how we detect this kind of layout bug? That we see there’s certain pattern here, which marks a box; it’s [INDISTINCT] box and this better continues here. There are some spaces between it, some lines. I think this should be, you put expresses (ph) in an algorithm I haven’t tried it out yet but I think it’s good. Okay, another one. Here’s some text is [INDISTINCT]. It’s not at the vertical edge so, I’m not sure how to do about it I really would like to have an OCR or Optical Character Recognition, I believe and [INDISTINCT], so, I could detect text on my page. And it should not detect this one because it’s not complete. And so, I could compare it. But I haven’t found a good OCR, I believe. At least not in the Java open source world (ph) And the last example is size of that spacing here and here. There’s an indentation problem here. I think so. I really have no idea how to go about this type of layout bugs. I hope you have some ideas. Don’t have an answer for every question. But it would be nice to takes those types of bug too, wouldn’t it? Okay, once again, back to the fighting layout bugs library. It’s really not more than a proof of concept at the moment. But I hope this doesn’t become something like FindBugs You know, FindBugs is a library, which takes typical programming bugs in Java classes In Fighting Layout Bugs should become library which finds typical layout bugs in web pages or something similar. Of course, contributions are welcome. If you have examples for layout bugs which are online; send them to the group, I would like to have a catalogue of typical layout bugs and if you have such a catalogue, we can think how we go about it to automatically detect those. If you have ideas for algorithms to detect automatically, layout bugs, let

us know. And if you want to contribute [INDISTINCT], feel free. It would be nice to–if I could start something and find that source. Of course, there are many open questions. I would like to try to answer to. First one is when should detectors be executed? First option is, of course, you can once again try to separate frontend test, which visits some pages, which you–you know your website best. You know which pages are special and need some attention Whoa. I’m so energetic. That might work but maybe it’s also [INDISTINCT] to execute simply expressing your frontend tests that you asserted that current page has no layout bugs. And then this [INDISTINCT] must have some of layout bugs, you figure out what detectors you want to use, to detect any layout bugs. On the third option, find out every time you get a new page or click on something, if you need something to change. Find out–find layout bugs every time. Well, it turns out–we have the test, that take some time, some seconds and the more detectors you use, the more time it will take. But on the other hand side, if you do this once in a night, in a nightly boot. Who cares if it takes one hour or two hours? And the last thing I want to mention is: What about false alarms? What if a detector says, “Oh, I detected a layout bug and if you look on it and say, “No, this is my design I want to look it like this.” But you can contribute a pitch (ph) to the fighting layout bugs, [INDISTINCT]. But then again, let’s look how FindBugs deals these sorts false alarms. The first option is they have a configuration file to exclude certain classes of bug detector So, we don’t get false alarm. I think it’s quite easy to do a configuration file, which has some URLs and some exclude patterns (ph), so, some detectors will not be executed by for these URLs. Maybe–another idea is to exclude certain areas of the screenshot. But as for page by only certain areas, maybe with some–some kind of coordinates or maybe with some pattern recognition; be able to specify a region (ph), the top left corner of region (ph), button right corner or maybe an [INDISTINCT], I’m not sure. And let’s start with some URL and just ignore detector for a certain page if you know if it is false alarm. Everything else is improvement. Why? Option number two, FindBugs has also this capability to scan the source files for command or sometimes, I think it also support annotations in the class files. That you specify–if this is some–your Java code, which I know creates FindBugs but [INDISTINCT] and I could imagine, who invents some kind of special HTML commands, which it will detect then process. And if it detects it, it just says, “Okay, I know wouldn’t want–don’t want me to report the layout bug here. Okay, so–quite easy isn’t it? Yeah, and–yeah that’s the last slide And before I take questions, I would like to thank you for your attention >> That is a very interesting presentation I must say, there’s going to be more political battle where you’re going to try and tell the developer, there’s a whole bunch of bugs and tell testers that they didn’t pick many of them up. One of the question that I want to ask is, you mentioned about testing these layout bugs across browser but I was not sure if you actually pointed out in your, you know, in one of these programs how you actually do that >> TAMM: I think this cause [INDISTINCT] problem, i’ts a special one, with all–as a layout bugs can be tested in one [INDISTINCT], how do we do about it? But if you–I guess if you use that driver, you have one API for all kinds of problems. So, we can take frontend test, and run it against Firefox and in the future, you run it against Chrome, run it against IE or IE would be a problem. I don’t [INDISTINCT] run it against Opera. And the same fronted test run on different browser and [INDISTINCT] the time to check, who knows, it could be the solution, I have to compare Damn. Okay, now, you can do it, but you have to somehow know which screenshot from one browser belongs to the browser. And if you have the screenshot of the same page, from two different browsers, then you must [INDISTINCT] us into the detector, which you haven’t yet what–if you have it, you can compare it I hope that satisfies somehow your answer I have a solution in project (ph) but I can’t imagine if it’s possible. The condition would be, to have a screenshot of the same page in one browser into the same page–screenshot of the same page from another browser. Then a clever compare [INDISTINCT]. I don’t think that’s it’s that hard >> Thank you for this presentation, I think it’s a–it was sort of Christopher Columbus solution to this–this is rather a clever solution to some point [INDISTINCT] and my question is I think you’ve mentioned assuming, upon result, assuming a lot web pages and scaling the text after three or four times So a lot of pages breaks or you can detect that as well, I guess, and detect it automatically as well? >> TAMM: If I understand the question correctly, what about detecting layout bugs when you zoom in the page. Only it’s zoomed on the one slide to show you how text looks magnified But, of course, this is a big problem, if you zoom a page as the text gets broaden, and sometimes this causes layout bugs, and I’ve seen this very often. I’m not sure of that type has brought by zooming, we should, shouldn’t we? It’s a part of it. Yeah? Yeah, maybe we can do it as keyword, shortcut for zooming. Yeah, of course, it’s an absolutely velvet front end test to zoom in that page and then triggers the layout bug detectors I–yeah, you should do this >> How is–with this work, because it’s something [INDISTINCT]canvass, if you got words in a canvass and some of the word’s are overlapping, is it–obviously, looking at it, visually, it’s a layout bug, but you can actually look into the canvass to see how things are >> TAMM: Yeah >> …be able to >> TAMM: Hmm, this wouldn’t work, would it? Only as the current detectors, which I have, who can only detect text overlapping edges, so as a question, yes, probably, it would detect overlapping texts? Hmm. Hmm. Hmm. I could imagine that you can–if you somehow know your webpage good enough, you can hide certain elements in this JavaScript and take screenshots. And if you know edges–yes, some special box, some special canvass which are safe. Just hid the text in there, take a screen shot that detects all the text, which is on the remains on the page and then flip around Hide all the other text and just place the text in the canvas, it takes a text pixels, then once again, do it with both end. If you see overlap, you have found the [INDISTINCT] >> Well, thank you. A couple of simple question, do you use it in your production process? >> TAMM: Not yet >> Then, no more question, I think. I just imagine how efficient–how much efficiency it has in real production process, you know, but we used to continuously–integration brought us to be detecting everyday, every single comment of our HTML guys, you know, and if we have several [INDISTINCT] or several [INDISTINCT] that commits everyday, so, can be highly detectible it or at can be rise in false alarms and so on? >> TAMM: Yeah, this is always the same, this new technology. This library has frightened you. I have developed it in the last months So… And if you introduce new technology, do it carefully step by step. So, what I proposed is start with some nightly Bluetooth that just checks some pages, and see how many false alarms you get in, how long the detection of layout bug takes. And then once you started, you can incrementally go from there by step by step by checking more pages or the tests more often. Let’s see how it scales. Maybe we can also televise the detectors to run on different missions. So, the time it takes to take layout bugs shortens. Yeah, but start as a little step and see how it goes from there, okay? >> First of all I would like to say thank you very much because you’re going to make my design. So I’m very, very, very happy So, thanks in advance. The second one is advance warning of a request that I’ll probably ask for you when anybody else mocking on this One of the biggest layout issues that I find in our application, and it’s a proprietor application, is we need to scan vouchers or some text or images, and then serve them up later on for somebody to validate and say, “This is good.” “This is not good.” And I find that you waste, I don’t know, 40% of the time telling the people who scan them to do a better scanning because if they scan it and look it up as a regular image, it’s fine. But if they serve it up on a browser, any browser, then, half of the time we have to send it by [INDISTINCT]. So, I guess, I’m fishing for ideas. Do you think we might be able to mock around with this so that we are testing images of images of images in order to try in–even if it just points out, you know, 10% of the errors, then I’ll have saved up a lot of time for that. So, you may have not have an answer right now but I’ll be following with you in quite a while >> TAMM: Yes. Okay. I will leave it overhand by this question. Test images for images, test images, hmm, meant as a–what I used here is a combination of image processing and some clever JavaScript and you can’t do this if you just have a screenshot. So, you need to have the excess to the–in a nice page in a browser, so it can execute some JavaScript to easily detect the layout bugs Yeah. That’s all I can I say, it’s a moment I–but let’s discuss this. I’m very interested to learn about your process of detecting layout bugs. And what kind of layout bugs do you have. Let us know. Maybe we will find a solution >> So, there was a class of layout bugs that I didn’t see you mentioned in there. But I did–I did see demonstrated. On your Yahoo page, the–it has you picture and then the text box and the text box is below the picture when based on the styling of the text box, it should definitely been to the right. And I think that was, again, because the German text for the submit button ended up being too long in pushing the flow down. So, I guess, that would be a flow error. And the only way I could think to detect that would be some kind of rule somewhere that says, “This is supposed to be the right of this.” I’m just wondering if you think this library is going to evolve to cover those kinds of layout bugs as well? >> TAMM: Yeah. Short answer, no; long answer is–actually, I wanted to present a force technique and this is–the idea is taking the–inter-creating the design as the actual requirement, which you have to test. This means if you say it must flow to the right, this is a requirement you have to test. And I think this was–this library gets you as the starting point to build your own tests to validate your design requirements. Another example would be if you have a design, normally, you get it good, which says the main column is 200 pixels and the right column is 100 pixels or something like that and there’s been a space of 10 pixels between it. And we have a very good edge detector. Why don’t you apply and see if the edges matches our [INDISTINCT] define by the design. And, I think, maybe I hope, you can use some culprits from this library to actually verify your design requirements, but I don’t think it fits onto the library because its