Carroll, nicknamed "the Hawk" was selected to the National League All-Star team in 1971 and 1972. He led the National League in saves in 1972 with 37, and finished tied for fifth in the Cy Young Award voting. The 37 saves stood as a National League record until Bruce Sutter broke it in 1984 with 45 saves pitching for the St. Louis Cardinals.
Carroll's best seasons were with the Reds, who he pitched for from 1968 to 1975 which earned him a place in the Cincinnati Reds Hall of Fame.
Carroll pitched in three World Series for Cincinnati including the 1975 World Series which the Reds beat Boston 4 games to 3. Carroll starred in the 1970 World Series as he appeared in five of the six games, hurling 9 shutout innings with 11 strikeouts to buyou a staff that otherwise struggled with injuries and ineffectiveness against Baltimore. Carroll was the winning pitcher in the Reds' only victory against the Orioles. Overall, Carroll had a stunning 1.39 ERA in 22 postseason appearances, allowing just five earned runs in 32.3 innings.