Rudolph Schwarz was born on November 16, 1841, in Germany. Along with his brother John, he emigrated to America, arriving on October 25, 1855. John headed south and settled in Savannah, Georgia. Rudolph stayed in New York City where he met Julianne Huber. They were married on October 8, 1861, in New York.

On August 17, 1861, he enlisted in the New York 45th Infantry, Company B. This regiment was also known as the 5th German Rifles because it was composed almost entirely of German immigrants. He was mustered in on September 9, 1861. The New York 45th fought in the Battle of Cross Keys, the First Battle of Rappahannock Station, and the Battle of Fredericksburg, among other battles.

On July 1, 1863, the 45th was among some of the earliest Union units to arrive on the field during the Battle of Gettysburg. Rudolph was involved in heavy fighting and at one point captured some Confederate prisoners at McClean's barn. While getting the prisoners ready to transfer to a holding place in the Gettysburg College building, Rudolph noticed one looked familiar. It was his brother John whom he had not seen for years. The two brothers enjoyed a happy reunion. Later that day, Rudolph was killed and his remains left on the battlefield. John Schwarz escaped the battle uninjured.

The New York 45th suffered 224 casualties at Gettysburg.