Abstract
This thesis investigates Bayesian and frequentist procedures for challenging highdimensional estimation problems.
In a Gaussian sequence model, we study the Bayesian approach to estimate the common variance of the observations. A fraction of the means is known to be zero, whereas the nonzero means are treated as nuisance parameters. This model is nonstandard in the sense that it induces inconsistent maximum likelihood. We show a general inconsistency result: the posterior distribution does not contract around the true variance as long as the nuisance parameters are drawn from an i.i.d. proper distribution. We also show that consistency is retained by a hierarchical Gaussian mixture prior. For the latter, we recover the asymptotic shape of the posterior in the Bernsteinvon Mises sense and show it is nonGaussian in the case of small means.
In the nonparametric regression model, we study the Bayesian approach to the estimation of a regression function that is characterized by some underlying composition structure, parametrized by a graph and a smoothness index. This model is inspired by deep learning methods, which work well when complex objects have to be built from simpler features. In previous work, a frequentist estimator based on deep neural networks has been shown to be adaptive with respect to the underlying structure and achieve minimax estimation rates. We characterize the contraction rates of the posterior distribution arising from priors induced by the composition of Gaussian processes. With a suitable model selection prior, we show that the posterior achieves the minimax rates of estimation.
In the nonparametric leastsquares regression model, we study a frequentist approach to estimate the regression function and the standard deviation of the residuals. The dataset consists of i.i.d. observations contaminated by a small number of outliers, and heavytailed residuals. For the case of known standard deviation, robust medianofmeans procedures are available, and we extend them to the case of unknown standard deviation. In the sparse linear regression case, the medianofmeans estimator yields a robust version of the Lasso, whereas our method yields a robust version of the squareroot Lasso thanks to a scaleinvariance argument. We also provide an aggregated estimator achieving minimax convergence rates while being adaptive to the unknown sparsity level.
In a Gaussian sequence model, we study the Bayesian approach to estimate the common variance of the observations. A fraction of the means is known to be zero, whereas the nonzero means are treated as nuisance parameters. This model is nonstandard in the sense that it induces inconsistent maximum likelihood. We show a general inconsistency result: the posterior distribution does not contract around the true variance as long as the nuisance parameters are drawn from an i.i.d. proper distribution. We also show that consistency is retained by a hierarchical Gaussian mixture prior. For the latter, we recover the asymptotic shape of the posterior in the Bernsteinvon Mises sense and show it is nonGaussian in the case of small means.
In the nonparametric regression model, we study the Bayesian approach to the estimation of a regression function that is characterized by some underlying composition structure, parametrized by a graph and a smoothness index. This model is inspired by deep learning methods, which work well when complex objects have to be built from simpler features. In previous work, a frequentist estimator based on deep neural networks has been shown to be adaptive with respect to the underlying structure and achieve minimax estimation rates. We characterize the contraction rates of the posterior distribution arising from priors induced by the composition of Gaussian processes. With a suitable model selection prior, we show that the posterior achieves the minimax rates of estimation.
In the nonparametric leastsquares regression model, we study a frequentist approach to estimate the regression function and the standard deviation of the residuals. The dataset consists of i.i.d. observations contaminated by a small number of outliers, and heavytailed residuals. For the case of known standard deviation, robust medianofmeans procedures are available, and we extend them to the case of unknown standard deviation. In the sparse linear regression case, the medianofmeans estimator yields a robust version of the Lasso, whereas our method yields a robust version of the squareroot Lasso thanks to a scaleinvariance argument. We also provide an aggregated estimator achieving minimax convergence rates while being adaptive to the unknown sparsity level.
Original language  English 

Qualification  Doctor of Philosophy 
Awarding Institution 

Supervisors/Advisors 

Award date  13 Oct 2021 
Place of Publication  Enschede 
Publisher  
Print ISBNs  9789036552356 
DOIs  
Publication status  Published  13 Oct 2021 