Testing giant planet formation in the transitional disk of SAO 206462 using deep VLT/SPHERE imaging

Context. The SAO 206462 (HD 135344B) disk is one of the few known transitional disks showing asymmetric features in scattered light and thermal emission. Near-infrared scattered-light images revealed two bright outer spiral arms and an inner cavity depleted in dust. Giant protoplanetshave been proposed to account for the disk morphology. Aims: We aim to search for giant planetsresponsible for the disk features and, in the case of non-detection, to constrain recent planetpredictions using the data detection limits. Methods: We obtained new high-contrast and high-resolution total intensity images of the target spanning the Y to the K bands(0.95-2.3 μm) using the VLT/SPHERE near-infrared camera and integral field spectrometer. Results: The spiral arms and the outer cavity edge are revealed at high resolutions and sensitivities without the need for aggressive image post-processing techniques, which introduce photometric biases. We do not detect any close-in companions. For the derivation of the detection limits on putative giant planetsembedded in the disk, we show that the knowledge of the disk aspect ratio and viscosity is critical for the estimation of the attenuation of a planetsignal by the protoplanetary dust because of the gaps that these putative planetsmay open. Given assumptions on these parameters, the mass limits can vary from 2-5 to 4-7 Jupiter massesat separations beyond the disk spiral arms. The SPHERE detection limits are more stringent than those derived from archival NaCo/L' data and provide new constraints on a few recent predictions of massive planets(4-15 MJ) based on the spiral density wave theory. The SPHERE and ALMA data do not favor the hypotheses on massive giant planetsin the outer disk (beyond 0.6''). There could still be low-mass planetsin the outer disk and/or planetsinside the cavity. Based on observations collected at the European Organisation for Astronomical Research in the Southern Hemisphere under ESO programmes 095.C-0298 and 090.C-0443.