We present Ynet -- a scene-aware trajectory prediction model that factorizes uncertainty into its epistemic (goal related) and aleatoric (path related) factors that achieves SOTA on both short-term & long-term time horizons on Stanford Drone, ETH/UCY and InD datasets.