{"id":79,"date":"2022-07-19T19:55:16","date_gmt":"2022-07-19T19:55:16","guid":{"rendered":"http:\/\/summergeometry.org\/sgi2022\/?p=79"},"modified":"2022-07-19T19:55:16","modified_gmt":"2022-07-19T19:55:16","slug":"fixing-a-bug-in-png2poly-from-gptoolbox","status":"publish","type":"post","link":"https:\/\/summergeometry.org\/sgi2022\/fixing-a-bug-in-png2poly-from-gptoolbox\/","title":{"rendered":"Fixing a bug in png2poly from gptoolbox"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\">Many of the SGI fellows experienced an error using the <a href=\"https:\/\/github.com\/alecjacobson\/gptoolbox\/blob\/a5f2fc15ecaf2cf76363c0d767e9bd3eaec7c3a5\/mesh\/png2poly.m\">png2poly<\/a> function from the <a href=\"https:\/\/github.com\/alecjacobson\/gptoolbox\">gptoolbox<\/a>: when applied to the <a rel=\"noreferrer noopener\" href=\"https:\/\/github.com\/odedstein\/sgi-introduction-course\/blob\/main\/201_polylines\/data\/toronto.png\" data-type=\"URL\" data-id=\"https:\/\/github.com\/odedstein\/sgi-introduction-course\/blob\/main\/201_polylines\/data\/toronto.png\" target=\"_blank\">example picture<\/a>, the function throws an indexing error and in some cases makes Matlab crash. So, I decided to investigate the code.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The function png2poly transforms a png image into closed polylines (polygons) and applies a Laplacian smoothing operation. I found out that this operation was returning degenerated polygons, which later in the code was crashing Matlab.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">But first, how does Laplacian smoothing work?<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">I had never seen how a smoothing operation works, and I was surprised with how simple the idea of Laplacian smoothing is. It&#8217;s an iterative operation, and at each iteration the positions of the vertices are updated using local information, the position of neighbor vertices.<\/p>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"844\" height=\"296\" src=\"http:\/\/summergeometry.org\/sgi2022\/wp-content\/uploads\/2022\/07\/curve.png\" alt=\"\" class=\"wp-image-119\" srcset=\"https:\/\/summergeometry.org\/sgi2022\/wp-content\/uploads\/2022\/07\/curve.png 844w, https:\/\/summergeometry.org\/sgi2022\/wp-content\/uploads\/2022\/07\/curve-300x105.png 300w, https:\/\/summergeometry.org\/sgi2022\/wp-content\/uploads\/2022\/07\/curve-768x269.png 768w\" sizes=\"auto, (max-width: 844px) 100vw, 844px\" \/><figcaption>New position of vertices after one iteration (credit: <a href=\"https:\/\/graphics.stanford.edu\/courses\/cs468-12-spring\/LectureSlides\/06_smoothing.pdf\">Stanford CS 468<\/a>).<\/figcaption><\/figure>\n<\/div>\n\n<p>\\[p_i \\gets p_i + \\frac{(p_{i+1} &#8211; p_i)}{2} + \\frac{(p_{i-1} &#8211; p_i)}{2}\\]<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">In polylines, every vertex has 2 neighbors, apart from boundary vertices, to which the operation is not applied. For closed polylines, Laplacian smoothing converges to a single point after many iterations.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The Laplacian matrix \\(L\\) can be used to apply the smoothing for the whole polyline at once, and a Lambda parameter (0 \u2264 \u03bb \u2264 1) can be introduced to control how much the vertices are going to move in one iteration:<\/p>\n\n\n<p>\\[p \\gets p + \\lambda L p\\]<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The best thing about Laplacian smoothing is that the same idea pleasantly applies for meshes in 3D! The difference is that in meshes every vertex has a variable number of neighbors, but the same formula using the Laplacian matrix can be used to implement it.<\/p>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"631\" height=\"206\" src=\"http:\/\/summergeometry.org\/sgi2022\/wp-content\/uploads\/2022\/07\/3d.png\" alt=\"\" class=\"wp-image-109\" srcset=\"https:\/\/summergeometry.org\/sgi2022\/wp-content\/uploads\/2022\/07\/3d.png 631w, https:\/\/summergeometry.org\/sgi2022\/wp-content\/uploads\/2022\/07\/3d-300x98.png 300w\" sizes=\"auto, (max-width: 631px) 100vw, 631px\" \/><figcaption> (credit: <a href=\"https:\/\/graphics.stanford.edu\/courses\/cs468-12-spring\/LectureSlides\/06_smoothing.pdf\">Stanford CS 468<\/a>)<\/figcaption><\/figure>\n<\/div>\n\n\n<p class=\"wp-block-paragraph\">For more details on smoothing, check out this <a rel=\"noreferrer noopener\" href=\"https:\/\/graphics.stanford.edu\/courses\/cs468-12-spring\/LectureSlides\/06_smoothing.pdf\" target=\"_blank\">slide<\/a> from Stanford, from which the images in this post were taken. It also talks about ways to improve this technique using an average of the neighbors weighted by curvature.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">What about the bug?<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">The bug was the following: For some reason, the function that converts the png into polygons was generating duplicate vertices. Later in the code, a triangulate function is used on these polygons, and the duplicate vertices by themselves make the function crash. But even worse, when smoothing is applied to a polygon with duplicate vertices, strange things happen. Here&#8217;s an example of a square with 5 vertices (1 duplicated); after 3 iterations it becomes a non-simple polygon:<\/p>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"304\" src=\"http:\/\/summergeometry.org\/sgi2022\/wp-content\/uploads\/2022\/07\/iterations-1-1-1024x304.png\" alt=\"\" class=\"wp-image-208\" srcset=\"https:\/\/summergeometry.org\/sgi2022\/wp-content\/uploads\/2022\/07\/iterations-1-1-1024x304.png 1024w, https:\/\/summergeometry.org\/sgi2022\/wp-content\/uploads\/2022\/07\/iterations-1-1-300x89.png 300w, https:\/\/summergeometry.org\/sgi2022\/wp-content\/uploads\/2022\/07\/iterations-1-1-768x228.png 768w, https:\/\/summergeometry.org\/sgi2022\/wp-content\/uploads\/2022\/07\/iterations-1-1-1536x456.png 1536w, https:\/\/summergeometry.org\/sgi2022\/wp-content\/uploads\/2022\/07\/iterations-1-1-1200x357.png 1200w, https:\/\/summergeometry.org\/sgi2022\/wp-content\/uploads\/2022\/07\/iterations-1-1.png 1928w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n<\/div>\n\n\n<p class=\"wp-block-paragraph\">You can try to simulate the algorithm to see that it happens.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Also, the lambda parameter used was 1.0, which was too high, making small polygons collapse or generating new duplicate points, so I proposed 0.5 as the new parameter. For closed curves, Laplacian smoothing will converge to a single point, making the polygon really small after many iterations, which is also a problem for the triangulation function. In most settings, these converged polygons can be erased.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Some other problems were also found, but less relevant to be discussed here. A pull request was merged into the gptoolbox repository removing duplicate points and fixing the other bugs, and now the example used in the tutorial week should work just fine. The changes I made don&#8217;t guarantee that the smoothing operation is not generating self-intersection anymore, but for most cases it does.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Fun fact: There&#8217;s a technique for finding if a polygon has self-intersection called <a rel=\"noreferrer noopener\" href=\"https:\/\/en.wikipedia.org\/wiki\/Bentley%E2%80%93Ottmann_algorithm\" target=\"_blank\">sweep line<\/a>, which works in \\(O(n\\log n)\\)&#8212;more efficient than checking every pair of edges!<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">The things I learned<\/h2>\n\n\n\n<ul class=\"wp-block-list\"><li>Laplacian smoothing is awesome.<\/li><li>It&#8217;s really hard to build software for geometry processing that works properly for all kinds of input. You need to have complete control over degenerate cases, corner cases, precision errors, &#8230; the list goes on. It gets even harder when you want to implement something that depends on other implementations, that may by themselves break with certain inputs. Now I recognize even more the effort of professor <a href=\"https:\/\/www.cs.toronto.edu\/~jacobson\/\">Alec Jacobson<\/a> and collaborators for creating gptoolbox and making it accessible.<\/li><\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">Big thanks to Lucas Valen\u00e7a, for encouraging me to try and solve the bug, and to Dimitry Kachkovski, for testing my code.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Many of the SGI fellows experienced an error using the png2poly function from the gptoolbox: when applied to the example picture, the function throws an indexing error and in some cases makes Matlab crash. So, I decided to investigate the code. The function png2poly transforms a png image into closed polylines (polygons) and applies a [&hellip;]<\/p>\n","protected":false},"author":15,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[35],"tags":[31,32],"class_list":["post-79","post","type-post","status-publish","format-standard","hentry","category-tutorial-week","tag-gptoolbox","tag-laplacian-smoothing"],"_links":{"self":[{"href":"https:\/\/summergeometry.org\/sgi2022\/wp-json\/wp\/v2\/posts\/79","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/summergeometry.org\/sgi2022\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/summergeometry.org\/sgi2022\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/summergeometry.org\/sgi2022\/wp-json\/wp\/v2\/users\/15"}],"replies":[{"embeddable":true,"href":"https:\/\/summergeometry.org\/sgi2022\/wp-json\/wp\/v2\/comments?post=79"}],"version-history":[{"count":9,"href":"https:\/\/summergeometry.org\/sgi2022\/wp-json\/wp\/v2\/posts\/79\/revisions"}],"predecessor-version":[{"id":243,"href":"https:\/\/summergeometry.org\/sgi2022\/wp-json\/wp\/v2\/posts\/79\/revisions\/243"}],"wp:attachment":[{"href":"https:\/\/summergeometry.org\/sgi2022\/wp-json\/wp\/v2\/media?parent=79"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/summergeometry.org\/sgi2022\/wp-json\/wp\/v2\/categories?post=79"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/summergeometry.org\/sgi2022\/wp-json\/wp\/v2\/tags?post=79"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}