Background: Colorectal cancer (CRC) is a major cause of cancer-related mortality worldwide. It is the second leading cause of cancer death in men and women in Brunei Darussalam in 2017, posing a major burden on society. Methods: This retrospective cohort study (n = 1035 patients diagnosed with CRC in Brunei Darussalam from 1st January 2002 until 31st December 2017) aims to compare the overall survival rates of CRC patients (2002–2017), to compare survival rates between two study periods (2002–2009 and 2010–2017) and to identify prognostic factors of CRC. Kaplan-Meier estimator and log-rank tests were performed to analyse the overall survival rates of CRC patients. Multiple Cox regression was performed to determine the prognostic factors of CRC with adjusted hazard ratios (Adj. HRs) reported. Results: The 1-, 3- and 5-year survival rates of CRC patients are 78.6, 62.5, and 56.0% respectively from 2002 to 2017. The 1-, 3-, and 5-year survival rates of CRC patients for 2002–2009 are 82.2, 69.6, and 64.7%; 77.0, 59.1, and 51.3% for 2010–2017 respectively. A significant difference in CRC patients’ survival rate was observed between the two study periods, age groups, ethnic groups, cancer stages, and sites of cancer (p < 0.05). The Adjusted Hazard Ratios (Adj. HRs) were significantly higher in the 2010–17 period (Adj. HR = 1.78, p < 0.001), older age group (≥ 60 years) (Adj. HR = 1.93, p = 0.005), distant cancer (Adj. HR = 4.69, p < 0.010), tumor at transverse colon and splenic flexure of colon (Adj. HR = 2.44, p = 0.009), and lower in the Chinese(Adj. HR = 0.63, p = 0.003). Conclusion: This study highlights the lower survival rates of CRC patients in 2010–2017, Malays, older patients, distant cancer, and tumors located at the latter half of the proximal colon (transverse colon), and predominantly LCRC (splenic flexure, descending colon, sigmoid colon, overlapping lesion colon and colon (NOS), as well as the rectosigmoid junction and rectum (NOS)). Age, ethnicity, cancer stage, and tumor location are significant prognostic factors for CRC. These findings underscore the importance of public health policies and programmes to enhance awareness on CRC from screening to developing strategies for early detection and management, to reduce CRC-associated mortality.