The digitization of health care has created vast repositories of electronic health records (EHRs) with significant potential to accelerate medical research and improve patient outcomes. However, this potential remains largely untapped due to stringent privacy regulations (eg, HIPAA [Health Insurance Portability and Accountability Act] and the GDPR [General Data Protection Regulation]) that restrict data sharing. Generative adversarial networks (GANs) [1] and newer tabular generators are widely used to create synthetic “shadow datasets” that can preserve important statistical characteristics while reducing direct data-sharing burden, although privacy risk must still be evaluated explicitly [2-5]. A more rigorous theoretical foundation for GAN-based modeling, including error and divergence analyses, can be found in the study of Hasan and Sang [6,7].

However, transitioning from research prototypes to real-world applications can be quite challenging, primarily due to concerns about clinical validity. Unlike natural images, where minor artifacts may be acceptable, clinical tabular records are governed by strict structural, physiological, and relational rules. A synthetic record may align with marginal distributions yet be impossible or clinically incoherent. Modern tabular generators such as Wasserstein GAN with gradient penalty (WGAN-GP) [8] and conditional tabular GAN (CTGAN) [9] therefore pose a practical question that statistical fidelity alone cannot answer: when can invalid outputs be repaired by inference-time selection, and when is the problem already baked into the learned support of the generator?

This discrepancy arises because standard generators optimize for distributional similarity rather than rule compliance. Bridging this “validity gap” has traditionally required one of 2 extremes: increasing architectural complexity by embedding constraints directly into the model or loss [10] or naive postprocessing, where invalid values are clipped or rounded after generation [11].

We study best-of-N sampling as a constrained selection strategy. For each desired output row, the method draws N candidates from the generator and returns the candidate with the minimum violation score. This mechanism is simpler than retraining and less destructive than clipping; however, it is not classical rejection sampling and cannot create valid samples when the generator assigns zero probability mass to the feasible region. We evaluate best-of-N across 3 datasets and 2 generator families and explicitly compare empirical success with the joint-validity probability p_valid=P(V(X)=0). This paper makes the following contributions:

All codes, constraint checkers, and the full evaluation pipeline are openly available on GitHub [12].

The goal of this study is to evaluate a data synthesis pipeline that produces clinically valid samples without compromising privacy or downstream usefulness. Our approach consists of three stages: (1) rigorous problem definition via dataset-specific constraints, (2) training a tabular generator backbone (WGAN-GP or CTGAN), and (3) inference-time post hoc sampling through random draws, naive repair, best-of-N selection, or strict rejection. We evaluate our approach on 3 publicly available clinical datasets, summarized in Table 2. Each dataset was selected to represent a different tier of constraint complexity, ranging from simple univariate bounds to complex multivariate correlations. All datasets were partitioned into training (70%), validation (15%), and testing (15%) sets.

We used 2 tabular generator families. First, we trained a WGAN-GP [8] as a deliberately simple baseline to study inference-time selection under imperfect validity. Second, we trained CTGAN [9] as a stronger tabular benchmark to test whether the same selection rule remains useful when the base model better captures mixed-type tabular structure. WGAN-GP models were trained with spectral normalization and a small soft-constraint penalty (λ_constraint=0.1); CTGAN was trained with its standard conditional sampling mechanism. Additionally, a purely unconstrained WGAN-GP baseline (λ=0, no soft-constraint penalty) was included to isolate the contribution of inference-time selection from that of the penalty term.

Best-of-N operates as a bounded-budget selection filter. For each desired synthetic record, we draw N candidate samples from a trained generator and return the candidate with the smallest violation score. This is not strict rejection sampling: the algorithm always returns the lowest-scoring candidate, even if all N candidates are invalid. Textbox 1 formalizes this procedure. We evaluated N ∈ {8, 16, 128} to capture bounded-budget, diminishing-returns, and large-budget regimes. For comparison, we also evaluated (1) random sampling, which draws directly from the generator without any filtering; (2) naive postprocessing, which applies deterministic value clipping, rounding, or one-hot repair after generation; and (3) strict rejection sampling, which repeatedly draws until V(x)=0, used to distinguish ranking-selection from true acceptance filtering.

The constraint checker computes per-record violations in 4 categories: structural or gender encoding, physiological range, relational correlation, and logical consistency. The default score is an equal-weight sum of indicator functions:

V(x)=Vgender(x)+Vphysio(x)+Vcorr(x)+Vlogic(x)(1)

For stroke, V_gender enforces one-hot gender consistency: |gender_male+ gender_female − 1| > 0.1 or out-of-range category values. For cardiovascular data, V_corr enforces the relational rule ap_hi > ap_lo + 10 mm Hg, and V_gender enforces valid encoding of the binary/ordinal gender field. For diabetes, V_corr encodes the dataset-specific logical inconsistency between diabetes status and self-reported general health. Equal weights are used because the primary objective is feasibility rather than ranking clinical severity; severity-weighted and no-gender ablations were also evaluated on stroke and cardiovascular data to assess robustness.

Let p_valid = P(V(X) = 0) denote the joint probability that a random sample from the trained generator satisfies all constraints simultaneously. Under independent draws:

P(at least 1 valid sample)=1−(1−pvalid)N(2)

We estimated p_valid empirically from raw generator samples:

p^valid=1M∑i=1M1{V(xi)=0}, xi∼Pmodel(3)

This yields the central distinction of the paper: when p_valid=0, neither best-of-N nor strict rejection can recover valid rows; when p_valid is moderate, empirical gains should follow the formula above.

All experiments were conducted in Python with PyTorch, Pandas (NumFOCUS, Inc), NumPy (NumFOCUS, Inc), SciPy (NumFOCUS, Inc), and scikit-learn (INRIA; French Institute for Research in Computer Science and Automation). For each dataset, we partitioned the data into training (70%), validation (15%), and test (15%) subsets and generated 500 synthetic samples per method for evaluation. WGAN-GP models used early stopping and a soft-constraint penalty of λ_constraint = 0.1; CTGAN was trained with its default conditional tabular formulation. Statistical uncertainty due to sampling variance was estimated using bootstrap resampling (B=1000 iterations). 95% CIs for violation rates are reported as Wilson score intervals based on 500 generated samples. Because algorithmic constraint matching is deterministic and inherently bounded by the 0/1 rule, formal P values are omitted; instead, we report absolute counts and empirical percentages throughout.

We report wall-clock time per generated sample for all main methods. For the mechanism analysis, we additionally report the expected time per valid sample, which is more informative than a kept/generated ratio when N is fixed by design. If a single draw takes an expected time t_g, then strict rejection has an expected cost E[T_reject]=t_g/p_valid per valid sample, while bounded-budget best-of-N has approximate cost E[T_BoN(N)] ≈ Nt_g / [1 − (1 − p_valid)^N]. These expressions explain why rejection fails when p_valid=0, and why large N can become inefficient after validity saturates, as illustrated by the costs reported above.

The datasets used in this study were obtained from publicly available repositories on University of California Irvine Machine Learning and Kaggle. All records are deidentified; therefore, this study did not involve human participants and was exempt from institutional review board review. The original data collection for each dataset was conducted under institutional ethical oversight, and the public release terms explicitly permit unrestricted secondary analysis; no additional informed consent was required for this study. Privacy preservation was empirically verified using membership inference attack (MIA) metrics (MIA AUC [area under the receiver operating characteristic curve] of approximately 0.50 across all methods).

The framework targeted 4 dimensions: validity (any-violation rate, target 0%), fidelity (Kolmogorov-Smirnov [KS] statistic, lower is better; correlation preservation, target≥0.90), utility (TSTR [train-on-synthetic-test-on-real] AUC, target≈real-data AUC), and privacy (MIA AUC≈0.50; distance to closest record [DCR] minimum ≥0). The computation methodology and formal targets for each metric are described in the following subsections.

Clinical validity was computed per record and summarized as the proportion of generated rows with at least one violation. Statistical fidelity was assessed using per-feature KS statistics and correlation preservation (the Frobenius norm of the difference between the synthetic and real Pearson correlation matrices). Utility was measured using TSTR AUC from a random forest classifier trained on synthetic data and evaluated on a held-out real test set. Privacy was assessed using MIA AUC and DCR summaries, with additional DCR-threshold concentration analysis for stroke and cardiovascular data. We use MIA as a standard empirical proxy for privacy attacks [28-31], following recent medical synthetic-data evaluations that recommend multimetric privacy reporting rather than a single attack score [5,18].

The updated experiments revealed 3 qualitatively different regimes for inference-time selection: a zero-valid support regime, a mostly valid regime, and a moderate-valid support regime. These regimes, rather than the nominal complexity of the constraint list alone, determined whether Best-of-N selection was useful.

Figure 1 summarizes violation rates across all 3 datasets and the full method stack, revealing 3 qualitatively distinct regimes. On stroke, WGAN-GP had no valid support mass (p_valid=0): random sampling, best-of-16, and best-of-128 all produced 500/500 (100%) invalid samples, and only CTGAN+best-of-16 reached 0/500 (0%) violations. On diabetes, WGAN-GP random sampling was already fully valid at 0/500 (0%) violations; the dataset illustrates the already-valid regime, where best-of-N shifts utility but not validity. On cardiovascular data, WGAN-GP random sampling produced 441/500 (88.2%, 95% Wilson CI 85.1%‐90.7%) invalid rows, best-of-16 reduced this to 65/500 (13.0%, 95% Wilson CI 10.3%‐16.2%), and best-of-128 achieved 0/500 (0%, 95% Wilson CI 0.0%‐0.8%)—matching the moderate p_valid=0.12 regime. Across all 3 datasets, CTGAN+best-of-16 achieved 0/500 (0%) violations. For methods with 0/500 violations (eg, CTGAN+best-of-16 across all 3 datasets, WGAN-GP random sampling on Diabetes), the Wilson upper bound is 0.8% at n=500; for the stroke WGAN-GP variants with 500/500 violations, the Wilson lower bound is 99.2%.

Unfiltered CTGAN random sampling achieved substantially lower violation rates than WGAN-GP across all datasets (stroke: 226/500, 45.2%; 95% Wilson CI 40.9%‐49.6%; diabetes: 14/500, 2.8%; 95% Wilson CI 1.7%‐4.6%; cardiovascular: 11/500, 2.2%; 95% Wilson CI 1.2%‐3.9%), with strong distributional fidelity (KS mean 0.07‐0.12, correlation preservation >0.90). CTGAN+best-of-16 then reduced violations to 0/500 on all 3 datasets, improving TSTR AUC on stroke from 0.3787 (CTGAN random) to 0.4772. The purely unconstrained WGAN-GP baseline (λ=0) produced per-sample violation rates of 100.0% (500/500) on stroke, 0.6% (3/500; 95% Wilson CI 0.2%‐1.7%) on diabetes, and 0.0% (0/500; 95% Wilson CI 0.0%‐0.8%) on cardiovascular disease. This indicates that the moderate-valid support regime (p_valid=0.12) is a characteristic of the penalized generator (λ=0.1) rather than the dataset itself. Adding the soft-constraint penalty (λ=0.1) to the WGAN-GP training loss degraded the model’s optimization stability, causing it to produce highly distorted distributions and a high violation rate (88.2%) on the cardiovascular dataset, whereas the unconstrained model (λ=0) avoided this training instability and directly learned the valid clinical ranges from the data. This comparison confirms that the soft-constraint penalty can degrade the baseline generator’s validity, making inference-time best-of-N selection necessary to recover validity in penalized models.

The stroke and cardiovascular results align closely with the joint-validity interpretation. For stroke WGAN-GP, the empirical valid support mass was p_valid=0.0, yielding predicted success of 0.0 for N=8, N=16, and N=128, with observed success also being 0.0 in all cases (Figure 2). For cardiovascular WGAN-GP, p_valid=0.12, and predicted success matched observed success almost exactly: 0.6404 vs 0.63 at N=8, 0.8707 vs 0.87 at N=16, and 1.0 vs 1.0 at N=128 (Figure 3). These results make the central point of the paper explicit: best-of-N cannot rescue a generator with zero-valid support mass, but when valid support mass is moderate, bounded-budget selection behaves as expected from the empirical value of p_valid.

Wall-clock sampling costs scaled approximately linearly with N: a single random draw required 0.35 to 0.56 ms, best-of-16 cost 6.86 to 9.68 ms, and best-of-128 cost 46.59 to 67.89 ms across the 3 datasets.

Sensitivity analysis showed that moderate reweighting did not materially change total cardiovascular violation rates, whereas removing the gender term did. For cardiovascular data, best-of-16 with equal weights yielded 11.0% violations in the sensitivity pool; best-of-16 with severity weighting also yielded 11.0% violations; and best-of-16 without the gender term increased violations to 53.5%. For stroke, all weighting profiles remained at 100.0% invalid, indicating that the problem was insufficient valid support rather than poor weighting.

Strict rejection sampling clarified the difference between ranking selection and true acceptance filtering. In stroke, rejection failed completely, producing 0 valid records across 5000 draws, consistent with the zero-support interpretation. In cardiovascular data, strict rejection was feasible: it produced 100 valid records from 724 draws (acceptance rate 0.1381) with 6.02 ms per valid sample. This was faster per valid sample than best-of-16 (10.77 ms/valid) and much faster than best-of-128 (63.79 ms/valid), indicating that best-of-N should not be framed as uniformly superior to rejection sampling when p_valid is already moderate.

Table 3 summarizes fidelity and utility for the main comparator methods. Figure 4 shows the utility-shift pattern in the diabetes dataset, where best-of-N substantially improves TSTR AUC, even though the validity is already 100%. CTGAN was consistently stronger than WGAN-GP on distributional fidelity, with much lower KS statistics and lower Frobenius error in the correlation matrix. Figure 5 illustrates this for the cardiovascular dataset. On the diabetes dataset, WGAN-GP best-of-N improved utility despite no validity gain: TSTR AUC increased from 0.5775 for WGAN-GP random sampling to 0.7149 for best-of-16 and 0.7258 for best-of-128. On the cardiovascular dataset, WGAN-GP best-of-128 reached 0.0% violations but remained poor for utility (TSTR AUC 0.50), whereas CTGAN+best-of-16 achieved both 0.0% violations and strong utility (TSTR AUC 0.7368). On the stroke dataset, CTGAN+best-of-16 improved TSTR AUC from 0.3787 for CTGAN random sampling to 0.4772 while preserving near-zero KS discrepancy in the gender columns. Naive clipping failed on utility across all datasets: on diabetes, it produced 500/500 (100%) violations due to binary feature distortion, rendering TSTR evaluation meaningless; on stroke and cardiovascular data, it maintained marginal utility (TSTR AUC=0.5000) at the cost of substantially higher KS statistics. These results confirm that naive clipping is not a viable substitute for constraint-minimizing selection when both validity and distributional fidelity are required.

MIA AUC remained close to 0.50 for all methods and datasets, with no clear evidence of successful membership inference. Table 4 reports exact MIA AUC and DCR mean values; Figure 6 shows the DCR proximity concentration pattern. For WGAN-GP on stroke and cardiovascular data, best-of-N changed concentration only slightly (stroke: 1.4%; cardiovascular: 0.4%). CTGAN+best-of-16 showed markedly stronger DCR concentration (22.6% for stroke; 100.0% for cardiovascular), even though MIA remained near chance throughout. These results support a cautious interpretation: no membership inference leakage was detected by MIA, but proximity-based concentration warrants additional monitoring when applying selection to stronger generators.

Three findings emerge from these experiments. First, best-of-N functions as a constraint-minimizing selector whose effectiveness depends on the generator’s valid support mass. The stroke WGAN-GP model had p_valid=0, and neither bounded-budget selection nor strict rejection across 5000 draws could recover valid samples. Second, when valid support mass is moderate (cardiovascular WGAN-GP, p_valid=0.12), empirical best-of-N behavior closely follows the theoretical success probability 1 − (1 − p_valid)^N. Third, CTGAN+best-of-16 delivered the strongest practical validity-fidelity-utility trade-off across all 3 datasets. Regarding privacy, MIA AUC remained near random guessing (0.50) throughout all experiments, which is reassuring. However, DCR-threshold concentration analysis revealed that stronger generators, combined with selection, can shift outputs closer to the training data in distance-based terms—a cautionary signal that privacy should not be assessed from MIA alone when applying selection to high-capacity generators.

These findings refine the usual comparison between architectural constraint handling and post hoc filtering. Best-of-N is not a universal replacement for architectural modeling, but it can serve as a practical inference-time wrapper when the base generator already places nontrivial probability mass near the feasible region. Therefore, our results sharpen rather than weaken the contribution: inference-time selection is effective in a subset of regimes that can be characterized empirically through p_valid, making the method more interpretable than a blanket “works across datasets” claim.

Generator quality matters more than candidate budget alone. WGAN-GP behaved differently across datasets: it failed completely on stroke data, was already valid on diabetes, and improved only gradually on cardiovascular data. CTGAN consistently produced low raw violation rates and responded well to best-of-16 selection. The primary recommendation from this study is therefore not simply to increase N but rather to pair bounded-budget selection with a generator that already models the feasible region reasonably well.

The rejection comparison clarifies an important boundary: when p_valid is 0, both rejection and best-of-N fail; when p_valid is moderate, strict rejection can be more efficient per valid sample than a large-budget best-of-N. Best-of-N’s practical value lies in its bounded computational overhead and compatibility with existing generators, not in guaranteed dominance over rejection sampling. On utility, mild best-of-N selection can shift the synthetic training distribution in a beneficial direction even when validity is already high (diabetes), but feasibility alone does not guarantee utility (cardiovascular: WGAN-GP best-of-128 achieved 0% violations, yet TSTR AUC remained 0.50).

First, this study is limited to 3 tabular clinical datasets and may not generalize directly to time series, imaging, or free-text medical data. Future work should assess generalizability across a wider range of EHR modalities. This may influence the generalizability of the 3-regime taxonomy, as datasets with richer temporal or unstructured constraint structures could yield qualitatively different p_valid distributions. Second, our diabetes validity definition follows the operational rule used in the code and is therefore a dataset-specific consistency heuristic rather than a universal clinical statement; the clinical significance of this rule should be validated by domain experts. As a result, the already-valid regime classification of diabetes may partly reflect the heuristic’s leniency rather than genuine generator validity, potentially overstating the method’s practical reach. Third, CTGAN concentration results indicate that stronger generators may require additional privacy diagnostics beyond MIA; practitioners should use DCR-threshold analysis alongside standard membership inference tests when deploying selection-augmented generators. The 100% DCR-threshold concentration for cardiovascular CTGAN +best-of-16 suggests that MIA AUC alone may substantially underestimate privacy risk, potentially giving practitioners a false sense of safety in high-stakes settings. Fourth, the paper reports one set of constraint weights as the default, and although sensitivity analyses reduced concern about arbitrariness, more comprehensive weighting strategies or lexicographic feasibility rules remain open for future work. Severity-weighted scoring could, in principle, produce different validity-utility trade-offs than those observed here, particularly in datasets where some constraint categories carry greater clinical consequence than others.

Future work should explore adaptive policies that select among random sampling, rejection sampling, and best-of-N based on the estimated p_valid; extend the same framework to diffusion- and language-model-based tabular generators [13,14]; and evaluate whether privacy-preserving training alters the interaction between valid support mass and inference-time selection.

Best-of-N selection is useful when treated as a bounded-budget feasibility filter rather than a universal repair mechanism. The present experiments show that its success is governed by the generator’s valid support mass: when that mass is 0, both selection and rejection fail; when it is moderate, empirical gains align with theoretical expectations; and when the generator is already strong, selection can remove residual violations at modest additional cost. In our setting, CTGAN+best-of-16 offered the strongest overall practical result across clinical validity, fidelity, utility, and privacy.