Statistical Learning for Structured Models: Tree Based Methods and Neural Networks
Freitag, 19.5.23, 12:00-13:00, Hörsaal II, Albertstr. 23b
Estimation in regression and classification problems which include low dimensional structures are considered. The underlying question is the following. How well do statistical learning methods perform for models with low dimensional structures? We approach this question using various algorithms in various settings. First, we introduce a new tree based algorithm we named random planted forest. It adapts particularly well to models which consist of low dimensional structures. We examine its performance in simulation studies and include some theoretical backing by proving optimal convergence rates in certain settings for a modification of the algorithm. A generalized version of the algorithm is included, which can be used in classification settings. Furthermore, we prove optimal convergence rates in a classification setting using neural networks. While non-optimal rates existed for this problem, we are the first to prove optimal ones.
Adaptive Testing: Bandits find correct answers fast
Freitag, 16.6.23, 12:00-13:00, Hörsaal II, Albertstr. 23b
Testing is the task of finding out which of several possible actions leads to the best outcome by repeatedly trying actions and observing their random effects. A company may want to find which web page A or B generates the most interaction with its clients. Clinical trials try to determine which drug quantity has the best efficiency-toxicity trade-off.\n\nIn the sequential testing framework, an agent repeatedly selects one of the actions and observes a random outcome. The agent wants to find the action with the best mean outcome as quickly as possible and with high certainty. A simple strategy is to try each action in turn until enough information is gathered. Bandit algorithms instead select their future actions based on past observations: they adapt to the data as it comes. This adaptive behavior makes them stop faster.
Toolbox for the Analysis of Motor Dynamics during Unrestrained Behavior
Freitag, 23.6.23, 12:00-13:00, Hörsaal II, Albertstr. 23b
Movement is the primary means of an organism interacting with its environment. To study neural processes underlying movement, neuroscientists often pursue the reductionist approach of reducing the variability of the task to a few controllable factors. Controlled but ethologically artificial paradigms delineate animal behavior into individual variables. Strongly limiting the animal's behavior via head-fixation reduces the number of uncontrolled variables in the design. However, it also limits our ability to understand the naturalistic dynamics of movement. Recent studies indicate that signals related to motion are spread throughout the whole brain, even in head-restrained animals. Spontaneous movements outside the task content influence the ongoing neural activity. These findings highlight the enormous contribution of behavior to neural activity and indicate that our interpretation of ongoing processes might be confounded. Thus, there is a drive in neuroscience to study neural processes in more naturalistic environments and freely moving conditions. Measuring animal behavior in such situations is not straightforward. Furthermore, many scientific tools for electrophysiology and optogenetic modulation were initially developed for acute experiments and need to be adopted for chronic, freely moving use. In this work, we developed multiple complementary tools to help study neural processes in freely moving animals. We designed and characterized different multifunctional techniques for combining electrophysiology and optogenetics. A fluidic probe could deliver viral vectors to the recording site; a multifiber approach enabled ultra-precise 3D interrogation of neural circuits. To measure unconstrained movements, we developed FreiPose, a versatile framework to capture 3D motion during freely moving behavior, and combined the movement tracking of rats with electrophysiology. Using a modeling strategy, we described the ongoing neural activity as a combination of simultaneous multiplexed coding of multiple body postures and paw movement parameters. A virtual head-fixation approach was devised of those models to distinguish paw movements from general movement information. Using encoding models of neural activity, we clamped body and head movements to obtain the impact of the paw movements on neuronal activity. Consequently, a large fraction of neurons in the motor cortex was uncovered to be tuned to paw trajectories. This tuning was previously masked by the influence of the varying body posture information. We conclude that measuring the movements of freely moving animals is an essential step toward understanding the underlying neural dynamics. Adding precise descriptions of ongoing behavior into computational models of neural activity will enable us to describe motifs of neural population activity related to sensorimotor integration and decision-making.
Evaluation of risks under dependence uncertainty
Freitag, 30.6.23, 12:00-13:00, Hörsaal II, Albertstr. 23b
Generalized Hoeffding-Fréchet functionals aiming at describing the possible influence of dependence on functionals of a statistical experiment, where the marginal distributions are fixed, have a long history. The problem of mass transportation can be seen as a particular case of this problem with two marginals and a linear functional induced by a distance. In the first part we give a review of some basic developments of this topic. We also describe several results for the solution for nonlinear functionals in the context of the analysis of worst case risk distributions. We show that these problems can be reduced to a variational problem and the solution of a finite class of (linear) masstransportation problems. \n\nIn the second part we review several approaches to improve risk bounds for aggregated portfolios of risks based on marginal information only. This endevour is motivated by the fact that the dependence uncertainty on the aggregated risks based on marginal information only is typically too wide to be acceptable in applications. Several methods have been developed in recent years to include structural and partial dependence information in order to reduce the model uncertainty. These include higher order marginals (method of reduced bounds), global variance or higher order moment bounds, partial positive or negative dependence restrictions and structural information given by common risk factors (partially specified risk factor models) or given by models with subgroup structures. Also an effective two-sided variant of the method of improved standard bounds has been developed.\n\nThe third part is devoted to some recent more detailed ordering results w.r.t. dependence orderings of relevant risk models making essential use of structural properties (like subgroup structure, graph structure or factor models) and on dependence properties of the models. The dependence structure of these models is given by *-products of copulas. Comparison results for *-products then allow to derive (sharp) risk bounds invarious subclasses of risk models induced by additional restrictions.\n\nSeveral applications show that these improved risk bounds may lead to results acceptable in praxis.
On a Mystery in Machine Learning
Freitag, 7.7.23, 12:00-13:00, Hörsaal II, Albertstr. 23b
In classical regression modelling, the complexity of the model, e.g. measured by the number of parameters, is smaller than the amount of training data. The prediction error exhibits a U-shaped behaviour. The (first) descent is due to decreasing bias, the ascent due to increasing variance. In modern machine learning, often the number of parameters by far exceeds the number of training data points. Intuitively, one could expect that the prediction error explodes with increasing model complexity due to overfitting. Belkin et al. (2019) observed that this is not the case. Instead, the prediction error decreases again when surpassing a certain threshold in model complexity, in some case even below the minimum of the classical, U-shaped regime. A phenomenon the authors denominated as double descent. To understand double descent, we study the simplest setting of linear regression and show that it can be explained by investigating the singular values of the design matrix. Finally, we give an outlook for the non-linear model setting.\n\nBelkin, M.; Hsu, D.; Ma, S.; Mandal, S.: Reconciling modern machine-learning practice and the classical bias–variance trade-off. In: Proceedings of the National Academy of Sciences 116 (2019), jul, Nr. 32, 15849–15854.
A Principal-Agent Framework for Optimal Incentives in Renewable Investments
Freitag, 14.7.23, 12:00-13:00, Hörsaal II, Albertstr. 23b
We investigate the optimal regulation of energy production reflecting the long-term goals of the Paris climate agreement.\n\nWe analyze the optimal regulatory incentives to foster the development of non-emissive electricity generation when the demand for power is served either by a monopoly or by two competing agents. The regulator wishes to encourage green investments to limit carbon emissions, while simultaneously reducing intermittency of the total energy production. We find that the regulation of a competitive market is more efficient than the one of the monopoly as measured with the certainty equivalent of the Principal’s value function. This higher efficiency is achieved thanks to a higher degree of freedom of the incentive mechanisms which involves cross-subsidies between firms. A numerical study quantifies the impact of the designed second-best contract in both market structures compared to the business-as-usual scenario. In addition, we expand the monopolistic and competitive setup to a more general class of tractable Principal-Multi-Agent incentives problems when both the drift and the volatility of a multi-dimensional diffusion process can be controlled by the Agents. We follow the resolution methodology of Cvitanić et al. (2018) in an extended linear quadratic setting with exponential utilities and a multi-dimensional state process of Ornstein-Uhlenbeck type. We provide closed-form expression of the second-best contracts. In particular, we show that they are in rebate form involving time-dependent prices of each state variable.
Segmentation and Estimation of Change-Points
Donnerstag, 14.9.23, 12:00-13:00, Raum 404, Ernst-Zermelo-Str. 1
We consider the problem of segmentation of (usually normal) observations according to changes in their mean. Changes can occur continuously, e.g., a change in the slope of a regression line, or discontinuously, e.g., a jump in the level of a process.Theoretical results will be illustrated by applications to copyn umber changes, historical weather records, and COVID-19--daily incidence, wastewater analysis, and excess deaths. Sequential detection, confidence regions for the change-points, and difficulties associated with dependent observations will also be discussed.\n\nAspects of this research involve collaboration with Fang Xiao, Li Jian, Liu Yi, Nancy Zhang, Benjamin Yakir, Keith Worsley, and Li (Charlie) Xia.