High specific surface area activated carbon was prepared by improving the process conditions of phosphoric acid activation and hulless barley straw and plain wheat straw as raw materials. The effects of the activation time on the pore structure and specific surface area of two types of activated carbon were investigated. The results revealed that soaking straws in the high concentrations of H3PO4 for 0.5 h to ensure complete soaking and then removing the straws from H3PO4 solution to activate was beneficial to the infiltration of H3PO4 in the raw material, and there was an increase in the surface area of activated carbon. The largest specific surface area of activated carbon prepared from plain wheat straw and hulless barley straw was 1524 m2/g and 1885 m2/g, respectively. Thermogravimetric analysis and scanning electron microscopy showed that the higher cellulose content, higher hemicellulose content, and smaller fiber morphology in hulless barley straw compared with wheat straw were the main reasons for the more abundant pore structure and higher specific surface area of the activated carbon.