Categories
Research

Do block-sparse featurizers find concept manifolds in sentence embeddings?

SGI Fellow: Pietro Palombini
Mentor: Ankita Shukla
Project volunteer: João Teixeira

TL;DR. Fel et al. (2026) find that block-sparse featurizers (BSFs) recover continuous concepts in vision models, such as the full circle of curve orientations in InceptionV1. We apply Fel et al.’s tests to sentence embeddings. These tests do not provide clear evidence that blocks recover concept manifolds in the embeddings we studied.

  • On text, blocks behave like finer clusters. The coordinates inside a block separate the categories that share it, raising category F1 on DBpedia from 0.54 to 0.79. But k-means clusters do about as well, and a few SAE features selected with labels do better on two of three datasets.
  • On synthetic circles, block stable ranks stay between 2.1 and 2.8 as blocks grow. On text they keep rising, so text blocks do not show the fixed low dimension that Fel et al. take as evidence of concepts.
  • Blocks give shorter descriptions of text embeddings than SAEs only at a loose reconstruction tolerance. The gain comes from identifying a few blocks instead of many features, and on text the blocks need more bits for everything else.
  • Moving a query along block directions retrieves fewer of a target’s neighbors than moving it along the difference between the two neighborhoods’ means.

1. What blocks are supposed to recover

Sparse autoencoders (SAEs) reconstruct activations as sparse combinations of learned directions (Bricken et al., 2023, and Gao et al., 2025). Some concepts vary continuously, such as the orientation of a curve. When a model’s representations of such a concept vary smoothly with it, they trace a concept manifold, which can span several dimensions. Previous work has found circular representations of the days of the week (Engels et al., 2025) and curved representations of character counts (Gurnee et al., 2025) in language models. A single direction cannot reconstruct such variation. BSFs instead reconstruct an embedding from a few active blocks of directions,

$$
\hat x = \sum_{g \in S(x)} D_g z_g,
$$

where \(S(x)\) is the set of active blocks and \(z_g\) are the coordinates of block \(g\). In the BSFs we use, the columns of each \(D_g\) are orthonormal. Our SAE baseline is the same architecture with blocks of size one. BSFs assume that embeddings are sparse sums of points on low-dimensional manifolds (Bhalla et al., 2026, and Fel et al., 2026).

Fel et al. (2026) report the following findings on vision models:

  • On synthetic sums of circles and other manifolds, blocks reconstruct the individual manifolds with mean \(R^2\) of 0.93 to 0.97, while the SAE features that best fit each manifold reach about 0.53.
  • A block represents the full circle of curve orientations in InceptionV1, which neurons and SAE features divide among separate detectors.
  • A block tracks scene luminance in DINOv3.
  • Moving within a block steers the images that SDXL, a diffusion model, generates.
  • At a fixed reconstruction tolerance, block codes describe DINOv3 activations in fewer bits than SAE codes.
  • Fel et al. report mean stable ranks between 2 and 4 even for blocks with 16 coordinates, and interpret this as evidence for concepts with two to four dimensions.

Fel et al. suspect that language fits the BSF assumption poorly, and O’Neill et al. (2024) find no irreducible multi-dimensional structure in the groups of SAE features they inspect on embeddings of arXiv abstracts. In GPT-2, on the other hand, one block of a block-sparse SAE responds to days, months, seasons and years (Dalili & Mahdavi, 2026). Our text datasets provide category labels but no known coordinates along a concept manifold, so we measure what blocks encode with the labels and test Fel et al.’s other evidence on synthetic circles.

2. How blocks organize categories

We use 20 Newsgroups (Lang, 1995), arXiv titles and abstracts (Schopf et al., 2024), and the opening paragraphs of Wikipedia articles with their DBpedia classes (Lehmann et al., 2015), with 19, 113 and 214 topic or entity-type categories. Dictionaries are trained on embeddings from all-mpnet-base-v2, an MPNet model (Song et al., 2020) fine-tuned with the Sentence-Transformers library (Reimers & Gurevych, 2019). We match BSFs and SAEs in both the total number of coordinates and the number active per document. Unless stated otherwise, BSFs have about one block per category, each with three coordinates, and two blocks active per document. Results average five seeds.

For each category, we predict membership from a block’s on/off state or, as Fel et al. do for image classes, from a logistic classifier on its coordinates. We compare blocks with untrained random blocks, with k-means clusters read out through three principal components each, and with single SAE features or groups of up to three. The clusters show what a partition of the space achieves without a dictionary. For every method, labels from training documents choose which unit reads out each category. For SAE groups, the labels also choose which features to combine, whereas BSF training groups directions without labels, so these groups test whether a few features chosen with labels can match a block learned without them. We evaluate all methods on test documents held out from training.

Category F1 for BSF blocks, SAE features and groups, k-means and random blocks.
Figure 1. Mean category F1 on test documents. Small dots show seeds and vertical marks their mean.

Reading a block’s coordinates instead of its on/off state raises F1 on all three datasets, most on DBpedia, from 0.54 to 0.79 (Figure 1). Block coordinates match a single SAE feature on 20 Newsgroups and exceed it on the other two datasets. Selected SAE groups score highest on 20 Newsgroups and arXiv. k-means clusters come within 0.06 of block coordinates on all three datasets.

A block active for several categories is a poor on/off detector for each of them, even when its coordinates separate them. The block in Figure 2 is active for posts about cars, motorcycles, electronics and other topics. Along the first principal component of its coordinates, car posts lie on one side and electronics posts on the other, with motorcycle posts between them.

Documents from several categories activating the same block, with example excerpts.
Figure 2. Test documents from 20 Newsgroups that activate the block most often active on training documents, projected onto two principal components of its coordinates. Colors mark the block’s three most frequent categories. Numbered points correspond to the excerpts below.

Dictionaries with more blocks give detectors with higher precision and lower recall (Table 1). This is consistent with each category being divided among several blocks, as in feature splitting in SAEs (Bricken et al., 2023, Chanin et al., 2025, and Leask et al., 2025). We test this by pooling the eight detectors with the highest training F1 for each category and predicting the category when any of them is active.

Table 1. On/off detection on 20 Newsgroups as the number of blocks grows.

Blocks Blocks per category Precision Recall F1
24 1.3 0.43 0.70 0.53
256 13 0.80 0.23 0.35
512 27 0.83 0.15 0.25
Category F1 before and after pooling eight BSF blocks or k-means clusters.
Figure 3. Mean category F1 with 256 BSF blocks or k-means clusters, for the best single detector of each category and a pool of its eight best. Dashed lines show a logistic classifier on the full embedding.

Pooling raises F1 on 20 Newsgroups from 0.35 to 0.65, against 0.75 for the full-embedding classifier, but lowers it on arXiv and DBpedia (Figure 3). With 256 units, 20 Newsgroups has about 13 units per category, arXiv 2 and DBpedia 1. Pooled clusters match or exceed pooled blocks on all three datasets.

3. Where category information lies in the embeddings

k-means comes close to blocks without training a dictionary, so we measure how much category information lies in the category means and in the variation around them. Let \(\mu_c\) be the mean of normalized training embeddings in category \(c\), and \(\bar\mu\) the mean of those centroids. Their centered span is the centroid subspace, which contains each category’s difference-in-means direction:

$$
S_{\mathrm{centroid}}
= \mathrm{span}\{\mu_c-\bar\mu\}_{c=1}^{C},
\qquad
\dim S_{\mathrm{centroid}}\le C-1,
$$ where \(C\) is the number of categories.

MPNet, MiniLM (Wang et al., 2020), E5 (Wang et al., 2022) and BGE (Xiao et al., 2024) show similar relative arrangements of category means. For each pair of models, we correlate the cosine similarities between centered category means, and the correlations range from 0.915 to 0.995 across the three datasets (Figure 4).

Category means for 20 Newsgroups in MPNet, MiniLM, E5 and BGE.
Figure 4. Category means for 20 Newsgroups, projected onto two principal components per model and aligned to MPNet by rotation or reflection. Colors group related newsgroups.

Projecting out the centroid subspace removes far more of each test query’s nearest neighbors than projecting out a random subspace of the same dimension (Figure 5). With MPNet, 72% of the neighbors remain on 20 Newsgroups and 16% on DBpedia, against 97% and 84% for the random subspace. The gap grows with the number of categories.

Neighbors retained after removing category means or random directions.
Figure 5. Fraction of each test query’s ten nearest neighbors that remain after removing the centroid subspace or a random subspace of the same dimension.

The variation around the means is also specific to each category. After removing the centroid subspace from MPNet embeddings of 20 Newsgroups, 20 principal components fit to a category explain 36% of the variance of its test documents around the category mean. Components fit to all training documents explain 20%, those of another category 11%, and random directions 3%. A clustering with local coordinates can use both kinds of information.

4. Recovering known circles

Separating categories does not show that a block reconstructs the variation within a concept, as the orientation block of Fel et al. does. We test this on synthetic data with known manifolds, as in the toy tests of Bhalla et al. (2026) and Fel et al. (2026). Twenty circles lie in random planes of a 768-dimensional space, and each sample sums one point from each of three random circles, with or without isotropic noise of standard deviation 0.05 per coordinate. We assign each circle a block, a single SAE feature and a group of up to three SAE features by their reconstruction of it on training samples. On independently generated test samples, reconstruction \(R^2\) measures how much of the circle’s contribution the unit reconstructs, and presence F1 measures whether the unit is active exactly when the circle is present. Untrained random blocks give a chance baseline. As a reference, we also reconstruct each circle from its known plane and the identities of the contributing circles.

Circle contributions and BSF reconstructions, with reconstruction R² and presence F1 across methods.
Figure 6. At the top, a test input combines circles 1, 2 and 3, and its BSF outputs include a good reconstruction, a poor one and an output from the block of absent circle 4. The middle panels project the outputs of two blocks into their circles’ planes, for test inputs that contain the circle. The bottom panels show \(R^2\) and presence F1 for each circle and seed, with method means as diamonds.

Without noise, 66 of 100 assigned blocks reconstruct their circle with \(R^2\) above 0.95. The others vary widely, from almost nothing to about 0.9, and the low-recovery block in Figure 6 outputs points near a line through its circle. Blocks therefore reach a mean reconstruction \(R^2\) of 0.82, compared with 0.48 for a single SAE feature, about what one direction can capture, and 0.99 for an SAE group (Figure 6). Blocks detect their circles more reliably than SAE groups, with presence F1 of 0.87 against 0.76. With noise, blocks still achieve lower reconstruction \(R^2\) but higher presence F1 than SAE groups. Scored as in Fel et al.’s toy test, which matches blocks to circles by when they are active and computes \(R^2\) only on inputs that contain the circle, blocks average 0.90 without noise and the best pair of SAE features 0.95. Fel et al. report 0.93 to 0.97 for blocks and 0.53 for a TopK SAE, in a denser setting of 128 manifolds of eight types in 128 dimensions.

Fel et al. interpret mean stable ranks between 2 and 4 as evidence for concepts with two to four dimensions. Table 2 compares text with the noisy circles, whose linear spans have dimension two, at matched numbers of total and active coordinates. We compute stable rank after centering each block’s codes over the inputs where it is active.

Table 2. Mean stable rank of centered block codes on test data, with 768 total and 48 active coordinates. Random blocks are untrained.

Block size 20 Newsgroups arXiv DBpedia Random blocks (20 Newsgroups) Circles
3 2.1 2.0 2.0 2.2 2.4
4 2.4 2.4 2.4 2.7 2.8
8 3.5 3.3 3.1 4.2 2.8
16 4.8 4.7 4.3 6.2 2.1

Blocks fit to the circles stay between 2.1 and 2.8 as blocks grow, ending at 2.1 at size sixteen (Table 2). The value does not determine a concept’s dimension exactly, but for the circles it stays within a narrow band. Text stable ranks instead rise steadily with block size, from about 2 at size three to between 4.3 and 4.8 at size sixteen, between the circles and untrained random blocks. Text blocks therefore do not show the fixed low dimension that Fel et al. take as evidence of concepts.

5. Description length

Fel et al. compare the bits needed to describe DINOv3 activations at a fixed reconstruction tolerance. We use a simpler estimate than Fel et al.’s. Following Ayonrinde et al. (2024), each active coefficient takes a fixed number of bits and we leave out the dictionary, and our residual term assumes isotropic Gaussian noise where Fel et al. use the residual’s covariance. With \(G\) blocks of size \(b\), \(k\) of them active, and coefficients quantized to \(B\) bits,

$$
L =
\mathop{\log_2{G \choose k}}\limits_{\mathrm{which\ blocks\ are\ active}}
+ \mathop{kbB}\limits_{\mathrm{their\ coefficients}}
+ \mathop{L_{\mathrm{residual}}}\limits_{\mathrm{remaining\ reconstruction\ error}}.
$$

Grouping shortens the index, the term Fel et al. single out as what distinguishes block codes from SAE codes. Identifying 12 active SAE features among 768 takes 86 bits, and identifying 4 active blocks among 256 takes 27.

Description length across block sizes at two reconstruction tolerances.
Figure 7. Estimated bits per test document at reconstruction tolerances \(R^2=0.5\) and \(0.8\), with 768 total and 12 active coordinates. Block size 1 is the SAE. Dashed lines show k-means.

At \(R^2=0.5\), some block sizes give shorter descriptions than the SAE on every dataset, and all do on arXiv and DBpedia (Figure 7). At \(R^2=0.8\), where coefficients and residual make up most of the length, the SAE is shortest on all three datasets. Each advantage of blocks is smaller than their reduction in index bits, so blocks take more bits than the SAE for coefficients and residual, and their shorter descriptions come from the index alone. With the same number of coordinates, blocks also achieve lower test \(R^2\), from 0.55 for the SAE to 0.43 for blocks of six on 20 Newsgroups.

6. Retrieval interventions

Fel et al. change generated images by moving activations within blocks. For text, we move a query toward a target document and measure how many of the target’s ten nearest neighbors the moved query retrieves, taking the best step size. Directions come from single units, full codes, pooled block or cluster spans, or the difference between the means of the target’s and the source’s neighborhoods (Marks & Tegmark, 2024). Apart from a random baseline and interpolation toward the target itself, all directions are built from the target’s neighbors, so the test measures how well each representation expresses a known displacement.

Retrieval of target neighbors after interventions, with interpolation as a reference.
Figure 8. Maximum recall of the target’s ten nearest neighbors, averaged over query pairs. Dashed lines show interpolation toward the known target vector.

The difference between neighborhood means recovers 0.73 to 0.74 of the target’s neighbors, against at most 0.43 for any dictionary or cluster direction (Figure 8). Wu et al. (2025) likewise find difference-in-means directions ahead of SAE features for detecting and steering concepts in Gemma 2. A single block outperforms a single SAE feature, but the full SAE code outperforms the full BSF code on all three datasets, and pooled k-means spans come within 0.03 of pooled blocks. These dictionaries were trained only for reconstruction, unlike the SAEs of Kang et al. (2025), which are trained with a retrieval objective.

7. Key takeaways

  • Compare blocks with k-means clusters and with small groups of SAE features chosen with labels before interpreting a block as a concept manifold. On text, clusters with local coordinates separated categories nearly as well as block coordinates, and the SAE groups did better on two of the three datasets.
  • Measure stable rank at several block sizes, and on synthetic data of known dimension, before reading it as a concept’s dimension. A single value did not reveal the dimension even for noisy circles. The trend separated text from circles, since text ranks kept rising with block size and circle ranks did not.
  • Compare description lengths at the same tolerance, and split each length into its terms. On text, blocks gave shorter descriptions only at a loose tolerance, and only because their index was shorter.
  • Use the difference between neighborhood means as a baseline for this kind of retrieval intervention. It retrieved more of a target’s neighbors than any dictionary or cluster direction we tested.

8. Conclusion

In our experiments on sentence embeddings, blocks behave like clusters with local coordinates, and Fel et al.’s tests do not provide clear evidence that they recover concept manifolds. A useful next step is to evaluate text with controlled variation within a concept and test whether block coordinates track that variation. Our dictionaries also train on up to about 40,000 documents per dataset, far fewer than the billions of token activations used for language-model SAEs (Gao et al., 2025, and Lieberum et al., 2024), so larger training sets could change these results.

References

  • Ayonrinde, K., Pearce, M. T., & Sharkey, L. (2024). Interpretability as Compression: Reconsidering SAE Explanations of Neural Activations with MDL-SAEs. NeurIPS 2024 Workshop on Machine Learning and Compression. arXiv:2410.11179
  • Bhalla, U., Fel, T., Rager, C., Feucht, S., Haklay, T., Wurgaft, D., Boppana, S., Kowal, M., Shyam, V., Merullo, J., Geiger, A., & Lubana, E. S. (2026). Do Sparse Autoencoders Capture Concept Manifolds? arXiv:2604.28119
  • Bricken, T., Templeton, A., Batson, J., et al. (2023). Towards Monosemanticity: Decomposing Language Models With Dictionary Learning. Transformer Circuits Thread. link
  • Chanin, D., Wilken-Smith, J., Dulka, T., Bhatnagar, H., Golechha, S., & Bloom, J. (2025). A is for Absorption: Studying Feature Splitting and Absorption in Sparse Autoencoders. NeurIPS 2025. arXiv:2409.14507
  • Dalili, S. A., & Mahdavi, M. (2026). Subspace-Aware Sparse Autoencoders for Effective Mechanistic Interpretability. arXiv:2606.06333
  • Engels, J., Michaud, E. J., Liao, I., Gurnee, W., & Tegmark, M. (2025). Not All Language Model Features Are One-Dimensionally Linear. ICLR 2025. arXiv:2405.14860
  • Fel, T., Kowal, M., Jacobs, M., et al. (2026). Structuring Sparsity: Block-Sparse Featurizers Capture Visual Concept Manifolds. arXiv:2606.25234
  • Gao, L., Dupré la Tour, T., Tillman, H., Goh, G., Troll, R., Radford, A., Sutskever, I., Leike, J., & Wu, J. (2025). Scaling and evaluating sparse autoencoders. ICLR 2025. arXiv:2406.04093
  • Gurnee, W., Ameisen, E., Kauvar, I., Tarng, J., Pearce, A., Olah, C., & Batson, J. (2025). When Models Manipulate Manifolds: The Geometry of a Counting Task. Transformer Circuits Thread. link
  • Kang, H., Wang, T., & Xiong, C. (2025). Interpret and Control Dense Retrieval with Sparse Latent Features. NAACL 2025. link
  • Lang, K. (1995). NewsWeeder: Learning to Filter Netnews. Proceedings of the Twelfth International Conference on Machine Learning, 331-339. link
  • Leask, P., Bussmann, B., Pearce, M., Bloom, J., Tigges, C., Al Moubayed, N., Sharkey, L., & Nanda, N. (2025). Sparse Autoencoders Do Not Find Canonical Units of Analysis. ICLR 2025. arXiv:2502.04878
  • Lehmann, J., Isele, R., Jakob, M., Jentzsch, A., Kontokostas, D., Mendes, P. N., Hellmann, S., Morsey, M., van Kleef, P., Auer, S., & Bizer, C. (2015). DBpedia: A large-scale, multilingual knowledge base extracted from Wikipedia. Semantic Web, 6(2), 167-195. doi:10.3233/SW-140134
  • Lieberum, T., Rajamanoharan, S., Conmy, A., Smith, L., Sonnerat, N., Varma, V., Kramár, J., Dragan, A., Shah, R., & Nanda, N. (2024). Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2. Proceedings of the 7th BlackboxNLP Workshop, 278-300. arXiv:2408.05147
  • Marks, S., & Tegmark, M. (2024). The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets. COLM 2024. arXiv:2310.06824
  • O’Neill, C., Ye, C., Iyer, K., & Wu, J. F. (2024). Disentangling Dense Embeddings with Sparse Autoencoders. arXiv:2408.00657
  • Reimers, N., & Gurevych, I. (2019). Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. EMNLP-IJCNLP 2019. arXiv:1908.10084
  • Schopf, T., Blatzheim, A., Machner, N., & Matthes, F. (2024). Efficient Few-shot Learning for Multi-label Classification of Scientific Documents with Many Classes. ICNLSP 2024, 186-198. link
  • Song, K., Tan, X., Qin, T., Lu, J., & Liu, T.-Y. (2020). MPNet: Masked and Permuted Pre-training for Language Understanding. NeurIPS 2020. arXiv:2004.09297
  • Wang, L., Yang, N., Huang, X., Jiao, B., Yang, L., Jiang, D., Majumder, R., & Wei, F. (2022). Text Embeddings by Weakly-Supervised Contrastive Pre-training. arXiv:2212.03533
  • Wang, W., Wei, F., Dong, L., Bao, H., Yang, N., & Zhou, M. (2020). MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers. NeurIPS 2020. arXiv:2002.10957
  • Wu, Z., Arora, A., Geiger, A., Wang, Z., Huang, J., Jurafsky, D., Manning, C. D., & Potts, C. (2025). AxBench: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders. ICML 2025. arXiv:2501.17148
  • Xiao, S., Liu, Z., Zhang, P., Muennighoff, N., Lian, D., & Nie, J.-Y. (2024). C-Pack: Packed Resources For General Chinese Embeddings. SIGIR 2024. arXiv:2309.07597
Categories
Research

Piecewise Developable Surfaces

Fellows: Yufan Hu, Robiul Islam Hannan, Kabir Sahni
Mentor: Yusuf Sahillioglu
Volunteer: Vivien van Veldhuizen

Introduction:

Developable surfaces are surfaces that can be unfolded onto a plane without stretching, compression, or other geometric distortion. They are relevant to applications in geometric design, digital fabrication, manufacturing, architecture, and related areas where three-dimensional geometry must be represented or fabricated from planar material. In continuous setting developable surface is charecterized by Gaussian curvature. The Gaussian curvature is formulated as:

K(v)=2π−∑Δ∋vθΔ(v)K(v) = 2\pi – \sum_{\Delta \ni v} \theta_{\Delta}(v) \quad
  • K>0 K>0 means surface is locally a bowl-like shape
  • K=0 K=0  means surface is locally flat in at least one direction 
  • K<0 K<0 means surface is locally a saddle-like shape

Our objective was to minimize non-flat curvature regions while preserving the geometric characteristics of the original surface. To address this problem, we formulated three vertex-based optimization techniques- normal one-ring, double-cone, and virtual-sphere– to relocate vertices through constrained directional searches while minimizing local angle-defect curvature. A complementary 3D-grid-based local search method explores possible vertex movements in the local normal and tangent directions. It then selects the movement that reduces the curvature while maintaining the surface geometry and preventing changes in triangle orientation.

Algorithm 1: Vertex relocation methods

Overview
All three methods restrict vertex motion to one-dimensional directions. However, only the double-cone and virtual-sphere methods explicitly sample and search along multiple rays, while the normal-constrained method jointly optimizes scalar displacements along vertex normals.

Normal joint optimizer
We use the angle defect to measure discrete Gaussian curvature at each interior vertex. A vertex is considered approximately developable when its absolute angle defect is below a prescribed threshold:

|Ωi|≤ϵ|\Omega_i|\leq\epsilon
  • Vertices satisfying ϵ<|Ωi|≤1\epsilon < |\Omega_i| \leq 1 are classified as moderate-defect vertices and selected as optimization targets.
  • Vertices whose initial angle defects satisfy |Ωi|>1|\Omega_i|>1 are treated as singular vertices and remain fixed, since the unavoidable curvature of the surface should be preserved somewhere.


Moving one vertex changes the corner angles and angle defects of its neighbors. Therefore, we jointly optimize each moderate-defect vertex together with its one-ring neighborhood. During one optimization step, each movable vertex is restricted to move along its current normal

𝐱i′=𝐱i+δi𝐧i\mathbf{x}_i’ = \mathbf{x}_i+\delta_i\mathbf{n}_i

Here, 𝐧i\mathbf{n}_i is the fixed vertex normal during the current optimization step, and δi\delta_i is the scalar displacement being optimized. This converts the vertex relocation problem from three variables per vertex to one variable per vertex, while still allowing neighboring vertices to move together.

The main objective penalizes only the part of the angle defect that exceeds the threshold:

Eκ=1|ℋ|∑i∈ℋ(max⁡(|Ωi|−ϵ,0)ϵ)2E_{\kappa} = \frac{1}{|\mathcal{H}|} \sum_{i\in\mathcal{H}} \left( \frac{\max(|\Omega_i|-\epsilon,0)}{\epsilon} \right)^2

The monitored set ℋ\mathcal{H} contains the movable vertices and their neighbors. A vertex contributes zero energy once it satisfies |Ωi|≤ϵ|\Omega_i|\leq\epsilon. Evaluating the energy over the neighborhood is important because reducing curvature at one vertex may transfer curvature to nearby vertices.
The complete objective combines this curvature term with regularization terms:

E=Eκ+λ0E0+λsEs+λpEp+λlEl+λeEeE = E_{\kappa} +\lambda_0E_0 +\lambda_sE_s +\lambda_pE_p +\lambda_lE_l +\lambda_eE_e

The remaining energy terms control how the mesh is deformed:

  • E0E_0 weakly encourages smaller angle defects
  • EsE_s preserves curvature around singular vertices
  • EpE_p limits vertex displacement
  • ElE_l encourages neighboring vertices to move smoothly, and
  • EeE_e penalizes relative edge-length changes.

Together, these terms prevent the optimizer from reducing angle defect by introducing excessive displacement, irregular local motion, or severe geometric distortion.

Figure1: Normal-constrained joint optimization.

Double-cone optimizer
The double-cone optimizer processes moderate-defect vertices sequentially, starting with the vertex having the largest absolute angle defect. At each target vertex, it samples movement directions from two cones centered on the positive and negative vertex-normal directions.
Using the local orthonormal frame (𝐧i,𝐭i1,𝐭i2\mathbf{n}_i,\mathbf{t}_{i1},\mathbf{t}_{i2}), an outward ray is defined as

𝐫j+=cj𝐧i+1−cj2(cos⁡ϕj𝐭i1+sin⁡ϕj𝐭i2),𝐫j−=−𝐫j+.\mathbf{r}_j^{+} = c_j\mathbf{n}_i+ \sqrt{1-c_j^2} \left( \cos\phi_j\,\mathbf{t}_{i1} + \sin\phi_j\,\mathbf{t}_{i2} \right), \qquad \mathbf{r}_j^{-}=-\mathbf{r}_j^{+}.


We sample 30 rays using a cone half-angle of (60∘60^\circ). The golden-angle sequence distributes the rays around the vertex normal.
Along each ray, candidate positions are parameterized by


𝐱i(t)=𝐱ik+t𝐫j,0≤t≤Li.\mathbf{x}_i(t)=\mathbf{x}_i^k+t\mathbf{r}_j, \qquad 0\leq t\leq L_i.


The search distance LiL_i limits both the current movement and the total displacement from the original mesh. We first evaluate five coarse intervals along each ray. If two consecutive samples contain a sign change in the angle defect, binary search is used to locate a near-zero-defect position.
A candidate is accepted only if it reduces the curvature objective and passes the geometric safety checks, including triangle orientation, area preservation, edge distortion, and cumulative displacement limits.
The cone construction is used only to generate search directions; the method does not compute the Shape Diameter Function.

Figure2: Double-cone ray search.

Virtual-sphere optimizer
The virtual-sphere optimizer uses the same search procedure, binary search, objective function, sequential updates, and geometric safety checks as the double-cone method. The only difference is how the ray directions are generated.
Instead of restricting the directions to two cones around the vertex normal, we use a Fibonacci sphere to distribute (m) rays over the entire unit sphere:
zj=1−2(j+1/2)m,ϕj=jπ(3−5)𝐫j=(1−zj2cos⁡ϕj,1−zj2sin⁡ϕj,zj)z_j=1-\frac{2(j+1/2)}{m}, \qquad \phi_j=j\pi(3-\sqrt{5}) \mathbf{r}_j= \left( \sqrt{1-z_j^2}\cos\phi_j,\, \sqrt{1-z_j^2}\sin\phi_j,\, z_j \right)
This allows the vertex to move in unrestricted normal, tangential, and oblique directions. Our current experiment uses (m=30) rays.

Figure3: Virtual-sphere ray search.

Algorithm 2: Grid-based method

Overview
This method proposes a local derivative-free optimization approach for reducing the non-flat curvatures of the triangle mesh while preserving the shape. We first describe the mathematical background of the algorithm and later explain how it reduces non-flat curvatures.

3D-grid local search with orientation protection

Step1:Gaussian curvature approximation
The algorithm uses the Gauss-Bonnet standard discretization formula for triangulated surfaces to estimate discrete Gaussian curvature at each vertex.

K(v1)=2π−∑Δ∋vθΔ(v)K(v1) = 2\pi – \sum_{\Delta \ni v} \theta_{\Delta}(v) \quad
K(v2)=π−∑Δ∋vθΔ(v)K(v2) = \pi – \sum_{\Delta \ni v} \theta_{\Delta}(v) \quad

Here,v1v1 and v2v2 denote the sets of interior and boundary vertices. θΔ(v)\theta_{\Delta}(v) \quadrepresents the interior angle of triangle Δ​ at vertex vv. θΔ(v)\theta_{\Delta}(v) \quad is calculated using the following formula:

θ=arctan2⁡(‖u×v‖,u⋅v)\theta = \operatorname{arctan2}(\Vert u \times v \Vert, u \cdot v)

u⋅vu \cdot v is the cross product proportional to the cos⁡θ\cos\theta and ∥u×v∥\|u \times v\| is proportional to the sin⁡θ\sin\theta .

Figure4: Gaussian curvature approximation.

Step2:
Local Coordinate Frame Construction:

u^=n^×e|n^×e|\hat{u} = \frac{\hat{n} \times e}{|\hat{n} \times e|}

v^=n^×u^\hat{v} = \hat{n} \times \hat{u}

To parameterize the local search space around a vertex, the algorithm constructs a customized local coordinate frame (n^,u^,v^)(\hat{n}, \hat{u}, \hat{v}) using the vertex unit normal vector n^\hat{n} and two orthogonal tangent vectors u^,v^\hat{u}, \hat{v}. ee is an arbitrary reference axis not parallel to n^\hat{n}

3D Grid Vector:

δ(tn,tu,tv)=tnn^+tuu^+tvv^,tn∈κS,tu,tv∈S,S=−η,0,η\delta(t_n, t_u, t_v) = t_n \hat{n} + t_u \hat{u} + t_v \hat{v}, \quad t_n \in \kappa S, \quad t_u, t_v \in S, \quad S = {-\eta, 0, \eta}

We introduce a 3D sampling grid using δ(tn,tu,tv)=tnn^+tuu^+tvv^\delta(t_n, t_u, t_v) = t_n \hat{n} + t_u \hat{u} + t_v \hat{v}the equation by choosing a linear combination of the basis vectors. Here, the step offsets are sampled from a uniform range bounded by a maximum displacement step η\eta. Crucially, the normal displacement tnt_n is scaled down by a damping factor κ\kappa which helps preserve mesh structure and fine geometric details.

Figure5: Local coordinate frame and 3D grid sampling.



Step4: Triangle normal calculation and orientation protection

𝐧Δ′=(p1−p0)×(p2−p0)‖(p1−p0)×(p2−p0)‖,𝐧Δ′⋅𝐧Δ0≥0.2∀Δ∋v\mathbf{n}’_{\Delta} = \frac{(p_1 – p_0) \times (p_2 – p_0)}{\Vert (p_1 – p_0) \times (p_2 – p_0) \Vert}, \quad \mathbf{n}’_{\Delta} \cdot \mathbf{n}^0_{\Delta} \ge 0.2 \quad \forall \Delta \ni v

The algorithm enforces a strict face-inversion constraint of all triangles Δ\Delta incident to the moving vertex vv to ensure that the vertex displacement does not compromise the topological integrity. For any candidate vertex position, the updated unit normal vector 𝐧Δ′\mathbf{n}’_\Delta is calculated via(p1−p0)×(p2−p0)(p_1 – p_0) \times (p_2 – p_0). The algorithm then evaluates the dot product between this candidate normal and the face’s original, unmodified normal
𝐧Δ0\mathbf{n}^0_\Delta. By requiring 𝐧Δ′⋅𝐧Δ0≥0.2\mathbf{n}’_\Delta \cdot \mathbf{n}^0_\Delta \ge 0.2, the algorithm imposes a lower strict bound that rejects any vertex update causing an adjacent triangle distortion. This function prevents mesh flipping, severe distortion, and self-intersection during local search.

Figure6: Orientation protection.



Step5:

Energy objective function:

E(pv)=|K(v)|+∑u∈N(v)[|K(u)|+W⋅1[|K(u)|>ε]]+λ‖pv−pv(0)‖E(p_v) = |K(v)| + \sum_{u \in N(v)} \left[ |K(u)| + W \cdot 1\big[ |K(u)| > \varepsilon \big] \right] + \lambda \| p_v – p_v^{(0)} \|

|K(v)||K(v)| minimizes the local Gaussian curvature defect at vertex vv to smooth irregularities. (∑u∈N(v))(\sum_{u \in N(v)}) evaluates Gaussian curvature changes on all 1-ring neighbour vertices u∈N(v)u \in N(v). [|K(u)|>ε]\mathbb[|K(u)| > \varepsilon] this indicator term applies a penalty ww if a move creates curvature spikes above threshold ε\varepsilon. The regularization term λ\lambda penalizes to prevent excessive surface drift.

Optimization:

pvopt=arg min⁡p∈{pvorig}∪{pvorig+δi}E(p)p_v^{\text{opt}} = \underset{p \in \{p_v^{\text{orig}}\} \cup \{p_v^{\text{orig}} + \delta_i\}}{\operatorname{arg\,min}} E(p)

The optimization evaluates candidate positions pp from the discrete 3D sampling grid set {pvorig+δi{p_v^{\text{orig}} + \delta_i}} alongside the original position pvorigp_v^{\text{orig}}. It greedily selects the candidate position pvoptp_v^{\text{opt}} that minimizes the energy objective E(p)E(p), ensuring the orientation protection constraint.

Figure7: Energy minimization and optimization.

Curvature Reduction and Geometry Preservation

We compare the four operators from two perspectives: curvature reduction and geometry preservation. Curvature-related metrics evaluate how effectively each operator reduces non-flat regions, while geometry-related metrics measure the corresponding deformation and mesh-quality loss.

Table 1: Curvature-state changes.

The transition counts are selected to distinguish corrected curvature violations from newly introduced ones. The final moderate-gray count measures the remaining non-flat vertices, while threshold energy describes the severity of these violations..

FamilyOperatorGray → Yellow ↑Yellow → Gray ↓Net reduction ↑Final moderate gray ↓Threshold energy ↓
Vertex-basedNormal one-ring432553771560.676
Vertex-basedDouble cone369383312023.659
Vertex-basedVirtual sphere349293202133.782
Figure8: From left to right: the original mesh, normal one-ring operator, double-cone operator, virtual-sphere operator, and 3D-grid operator.

Table2: Curvature state transition in Algorithm 2

We evaluated the proposed 3D grid based algorithm on five models namely Cheburashka, Fandisk, Stanford-bunny, Homer, Suzanne. The optimal threshold epsilon was set to 0.1 and the penalty was fixed at 50. The table summrizes the before and after curvature counts with the algorithm run time.

ModelEpsilonPenaltyFlat Vertices
(Before)
Non-Flat Vertices
(Before)
Flat Vertices
(After)
Non-Flat Vertices
(After)
Flat Vertex
Gain
Run time
Cheburashka0.15062933766411258+118308.198s
Fandisk0.150642748642946+258.310s
Stanford-bunny0.1503407154034444167+373410.793s
Homer0.15053136895526476+213535.254s
Suzanne0.150115350193272+78317.417s
Figure9: Visualization of the Cheburashka before(left) and after(right) applying the algorithm. Red, yellow, and blue color point out non-flat, flat, and regions converted non-flat to flat.
Figure10: Visualization of the Fandisk before(right) and after(left) applying the algorithm. Red, yellow, and blue color point out non-flat, flat, and regions converted non-flat to flat.

Table 3: Geometry preservation.
Normalized RMS displacement measures the overall surface change relative to the model size, while maximum edge change captures the worst local edge-length distortion. Minimum area ratio measures the strongest triangle compression, with values closer to one indicating better area preservation. Minimum triangle quality ranges from zero to one, where values near zero indicate thin or nearly degenerate triangles. Maximum normal change measures the largest local surface-orientation deviation, and the flipped-face count detects invalid face inversions.

FamilyOperatorNormalized RMS ↓Max edge change ↓Min area ratio ↑Min quality ↑Max normal change ↓Flipped faces
Vertex-basedNormal one-ring0.00569%24.71%0.3040.0084128.29°0
Vertex-basedDouble cone0.00373%19.14%0.3130.0139029.19°0
Vertex-basedVirtual sphere0.00407%23.97%0.5940.0139073.81°0
Grid-based3D grid0.01315%21.99%0.3510.0139230.00°0

Overall result:
The results indicate no operator dominates across all criteria. The double cone achieved the highest normalized RMS and maximum edge change, with a competitive minimum quality. The virtual sphere preserves triangle area best, but at the cost of a much larger maximum normal change. The normal one ring yields the smallest normal deviation but the lowest minimum quality and the largest edge change. The 3D grid is competitive in quality and normal change but has the highest RMS displacement. All the operators produced zero flipped faces.

Future directions:

  • Extending these algorithm for real time use will likely require cost reduction strategies such as principal component analysis (PCA), proper orthogonal decomposition (POD).
  • The evaluation covers geometric fidelity and mesh validity only. A systematic study of memory scaling and computational cost reduction strategies is a natural direction for future work.
  • Testing on more complex models is needed to check the generalizability of the algorithms.

Reference:

  • R. H. Byrd, P. Lu, J. Nocedal, and C. Zhu, A Limited Memory Algorithm for Bound Constrained Optimization, pp. 1190–1208, 1995. https://doi.org/10.1137/0916069
  • L. Shapira, A. Shamir, and D. Cohen-Or, Consistent Mesh Partitioning and Skeletonisation using the Shape Diameter Function, vol. 24, no. 4, pp. 249–259, 2008. https://doi.org/10.1007/s00371-007-0197-5
  • B. Keinert, M. Innmann, M. S\”anger, and M. Stamminger, Spherical Fibonacci Mapping, vol. 34, no. 6, article 193, 2015. https://doi.org/10.1145/2816795.2818131
  • D. Zint, R. Grosso, and F. Lunz, “Discrete mesh optimization on surface and volume meshes,” in Proc. 28th Int. Meshing Roundtable (IMR), 2019, pp. 159–174.
  • A. Nealen, T. Igarashi, O. Sorkine, and M. Alexa, “Laplacian mesh optimization,” in Proc. 4th Int. Conf. Computer Graphics and Interactive Techniques in Australasia and Southeast Asia (GRAPHITE ‘06), 2006, pp. 381–389.