get_frame<\/strong><\/code> and apply_frame<\/strong><\/code> functions. <\/p>\n\n\n\ndef forward(self, pnt_cloud):\n\t# compute frames by PCA\n\tframe, center = self.get_frame(pnt_cloud)\n\t# apply frame operations to re-centered point cloud\n\tpnt_cloud_framed = self.apply_frame(pnt_cloud - center, frame)\n\n\t# extract features of framed point cloud with PointNet\n\tpnt_net_feature = self.pnt_net(pnt_cloud_framed)\n\t# predict likelihood of classification to each category\n\tpred_scores = self.classify(pnt_net_features)\n\t# take the average of prediction scores over the 8 frames\n\tpred_scores_averaged = pred_scores.mean()\n\n\treturn pred_scores_averaged\n\n<\/pre><\/div>\n\n\nExperiment and Comparison<\/h2>\n\n\n\n
We chose Vector Neurons, a recent framework for invariant point cloud processing, as our experiment baseline. By extending neurons from 1-dimensional scalars to 3-dimensional vectors, the Vector Neuron Networks (VNNs) enable a simple mapping of SO(3) actions to latent spaces and provide a framework for building equivariance in common neural operations. Although VNNs could construct rotation-equivariant learnable layers of which the actions will commute with the rotation of point clouds, VNNs are not very compatible with the translation of point clouds.<\/p>\n\n\n\n
In order to build a neural network that commutes with the actions of the SE(3) group (rotation + translation) instead of the actions of the SO(3) group (rotation only), we will use Frame Averaging to construct the equivariant autoencoders that are efficient, maximally expressive, and therefore universal.<\/p>\n\n\n\n
Following the implementation described above, we trained a classification model on 5 selected classes (bed, chair, sofa, table, toilet) in the ModelNet40 dataset. We obtained a Vector Neurons classification model with the same PointNet backbone (input and feature transformation included) pre-trained on the complete ModelNet40 dataset. We tested the classification accuracy on both models with point cloud data of the 5 selected classes randomly transformed by rotation-only or rotation and translation. The results are shown in the table below.<\/p>\n\n\n\n\n![\"\"](\"http:\/\/summergeometry.org\/sgi2022\/wp-content\/uploads\/2022\/08\/Untitled-3-1024x970.png\")
Original Point Cloud of Chair<\/figcaption><\/figure>\n\n\n\n![\"\"](\"http:\/\/summergeometry.org\/sgi2022\/wp-content\/uploads\/2022\/08\/Untitled-1-3-1024x960.png\")
Rotation-only<\/figcaption><\/figure>\n\n\n\n![\"\"](\"http:\/\/summergeometry.org\/sgi2022\/wp-content\/uploads\/2022\/08\/Untitled-2-2-1024x948.png\")
Rotation and Translation<\/figcaption><\/figure>\n<\/figure>\n\n\n\n<\/p>\n\n\n\n<\/td> SO(3) Test Instance Accuracy<\/td> SE(3) Test Instance Accuracy<\/td><\/tr> Vector Neurons<\/td> 89.6%<\/td> 6.1%<\/td><\/tr> Frame Averaging<\/td> 80.7%<\/td> 80.3%<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\nThe Vector Neurons model was trained for a more difficult classification task, but it was also more fined tuned with a much longer training time and larger data size. The Frame Averaging model in our experiment, however, was trained with relatively larger learning steps and shorter training time. Although it is not fair to make a direct comparison between the two models, we can still see a conclusive result. As expected, the Vector Neurons model has good SO(3) invariance, but tends to fail when the point cloud is translated. The Frame Averaging model, on the other hand, performs similarly well on both SO(3) and SE(3) invariance tests.<\/p>\n\n\n\n
If the Frame Averaging model was trained with the same setting as the Vector Neurons model, we believe that it will have both the accuracy of SO(3) and SE(3) tests comparable with Vector Neurons\u2019 SO(3) accuracy, because of its maximal expressive power. Due to the limited resource, we cannot prove our hypothesis by experience, but we will provide more theoretical analysis in the next section.<\/p>\n\n\n\n
Discussion: Expressive Power of Frame Averaging<\/h2>\n\n\n\n
Besides the good performance in maintaining SE(3) invariance in classification tasks, we want to further discuss the advantage of frame averaging, namely its ability to preserve the expressive power of any backbone architecture.<\/p>\n\n\n\n
The Expressive Power of Frame Averaging<\/h4>\n\n\n\n
Let \\(\\phi: V \\rightarrow \\mathbb{R}\\) and \\(\\Phi: V \\rightarrow W\\) be some arbitrary functions, e.g. neural networks where \\(V, W\\) are normed linear spaces. Let \\({G}\\) be a group with representations \\(\\rho_1: G \\rightarrow GL(V)\\) and \\(\\rho_2: G \\rightarrow GL(W)\\) that preserve the group structure.<\/p>\n\n\n\n
Definition<\/strong><\/p>\n\n\n\nA frame is bounded over a domain \\(K \\subset V\\) if there exists a constant \\(c > 0\\) so that \\(||\\rho_2(g)|| \\leq c\\) for all \\(g \\in F(X)\\) and all \\(X \\in K\\) where \\(||\\cdot||\\) is the operator norm over \\(W\\).<\/p>\n\n\n\n
A domain \\(K \\subset V\\) is frame finite if for every \\(X \\in K\\), \\(F(X)\\) is a finite set.<\/p>\n\n\n\n
A major advantage of Frame Averaging is its preservation of the expressive power of the base models. For any class of neural networks, we can see them as a collection of functions \\(H \\subset C(V,W)\\), where \\(C(V,W)\\) denotes all the continuous functions from \\(V\\) to \\(W\\). We denote \\(\\langle H \\rangle = \\{ \\langle \\phi \\rangle | \\phi \\in H\\}\\) as the transformed \\(H\\) after applying the frame averaging. Intuitively, the expressive power tells us the approximation power of \\(\\langle H \\rangle\\) in comparison to \\(H\\) itself. The following theorem demonstrates the maximal expressive power of Frame averaging.<\/p>\n\n\n\n
Theorem<\/strong><\/p>\n\n\n\nIf \\(F\\) is a bounded \\(G\\)-equivariant frame over a frame-finite domain \\(K\\), then for any equivariant function \\(\\psi \\in C(V,W)\\), the following inequality holds<\/p>\n\n\n\n
\\(\\inf_{\\phi \\in H}||\\psi – \\langle \\phi \\rangle_F||{K,W} \\leq c \\inf{\\phi \\in H}||\\psi – \\phi||_{K_F,W} \\)<\/p>\n\n\n\n
where \\(K_F = \\{ \\rho_1(g)^{-1}X | X \\in K, g \\in F(X)\\}\\) is the set of points sampled by the FA operator and $c$ is the constant from the definition above.<\/p>\n\n\n\n
With the theorem above, we can therefore prove the universality of the FA operator. Let \\(H\\) be any collection of functions that are universal set-equivariant, i.e., for arbitrary continuous set function \\(\\psi\\) we have \\(\\inf_{\\phi \\in H} ||\\psi – \\phi||_{\\Omega,W} = 0\\) for arbitrary compact sets \\(\\Omega \\subset V\\). For bounded domain \\(K \\subset V\\), \\(K_F\\) defined above is also bounded and contained in some compact set \\(\\Omega\\). Therefore, we can conclude with the following corollary.<\/p>\n\n\n\n
Corollary<\/strong><\/p>\n\n\n\nFrame Averaging results in a universal SE(3) equivariant model over bounded frame-finite sets, \\(K \\subset V\\).<\/p>\n","protected":false},"excerpt":{"rendered":"
By Ruyu Yan, Xinyi Zhang Invariant Point Cloud Classification We experimented with incorporating SE(3) invariance into a point cloud classification model based on our prior study on Frame Averaging for Invariant and Equivariant Network Design. We used a simple PointNet architecture, dropping the input and feature transformations, as our backbone. Method and Implementation Similar to […]<\/p>\n","protected":false},"author":9,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[45],"tags":[],"class_list":["post-937","post","type-post","status-publish","format-standard","hentry","category-research"],"_links":{"self":[{"href":"https:\/\/summergeometry.org\/sgi2022\/wp-json\/wp\/v2\/posts\/937","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/summergeometry.org\/sgi2022\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/summergeometry.org\/sgi2022\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/summergeometry.org\/sgi2022\/wp-json\/wp\/v2\/users\/9"}],"replies":[{"embeddable":true,"href":"https:\/\/summergeometry.org\/sgi2022\/wp-json\/wp\/v2\/comments?post=937"}],"version-history":[{"count":10,"href":"https:\/\/summergeometry.org\/sgi2022\/wp-json\/wp\/v2\/posts\/937\/revisions"}],"predecessor-version":[{"id":1129,"href":"https:\/\/summergeometry.org\/sgi2022\/wp-json\/wp\/v2\/posts\/937\/revisions\/1129"}],"wp:attachment":[{"href":"https:\/\/summergeometry.org\/sgi2022\/wp-json\/wp\/v2\/media?parent=937"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/summergeometry.org\/sgi2022\/wp-json\/wp\/v2\/categories?post=937"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/summergeometry.org\/sgi2022\/wp-json\/wp\/v2\/tags?post=937"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}