This study examines how the chemical composition of ultramafic and mafic rocks effects their physicomechanical properties and therefore how influences the concrete strength of the produced concrete specimens. For this scope, ultramafic (Group I) and mafic rocks (Group II) derived from the Veria–Naousa and Edessa ophiolite complexes (Greece) were selected in order to identify their chemical composition and their engineering properties according to international standards. Additionally, representative rocks were used as concrete aggregates in order to produce concrete specimens, whereas their mechanical strength was calculated. A geochemical index (Ga) was proposed and correlated with the engineering properties of the examined rocks as well as with the widely used alteration degree LOI (loss on ignition). Correlation diagrams between engineering properties and the proposed geochemical index (Ga) have showed that these properties were strongly influenced by the alteration processes expressed via Ga index. More particularly, mainly serpentine in ultramafic and chlorite in mafic rocks, minerals indicators for the alteration of ultramafic and mafic rocks, respectively, seem to determine their engineering properties. Concerning the mechanical strength of the produced concrete specimens, the results have showed that the increasing values of Ga index negatively effect concrete strength.