Graphene oxide was prepared from pencil type 2B by modified Hummers method and used for coating of polyurethane sponge. The graphene oxide-modified polyurethane (GO/PU) sponge was applied as an adsorbent for oil-water separation, which was represented by the removal of diesel oil from water. The formation of graphene oxide and GO/PU was characterized by spectroscopic, imaging, and X-ray diffraction methods. The best adsorption performance of GO/PU was 71.2% efficiency within 10 s of contact time. It was demonstrated that the incorporation of graphene oxide enhanced the adsorption performance of the PU sponge which showed 54.9% efficiency diesel removal from water under the same experimental condition. The result also suggested that graphene oxide (GO) has a synergistic effect of superhydrophobicity and good mechanical performance in the GO/PU sponge.