Large trees (diameter at breast height ≥50 cm) play a vital role in providing ecological services in tropical forests and can store up to 50% of aboveground biomass. We expected that topography (as a nutrient and drought proxy) would affect large tree dynamics, biomass, diversity, and composition patterns. To test this, we established a 15-ha permanent plot at Andulau Forest Reserve (Brunei Darussalam, Borneo) with an elevation ranging from 42 m (swamp) to 92 m (ridge tops). All 559 large trees (131 species) were labeled, identified, and fitted with dendrometers. Stem density, basal area (BA), soil and litter properties, absolute diameter growth rate (AGR), and recruitment and mortality rates of large trees were determined for five elevational zones for the period 2016–2019. Stem density and BA of large trees were significantly higher in the upslope and ridge areas of the plot, and a clear shift in large tree species composition was found across the elevational zones, with the swampy area differing most from all others. However, AGR did not differ between elevational zones. Large tree mortality was balanced by recruitment, but sample size was too low to detect topographical patterns. Our findings suggest that the large tree community retained similar growth rates across soil water and nutrient gradient via a shift in species composition. If so, predicted declines in large tree productivity with climate change may, over time, also be offset by changes in species composition.