After signing Leon Best on Tuesday, former Republic of Ireland international Kelly is the next through the door at the Millers after a successful trial, penning a deal until the end of the season.
The 32-year-old, who has played in the Premier League for Tottenham, Birmingham, Fulham and Reading, has been without a club since leaving the Royals in the summer.
He came to the club a fortnight ago to train and has impressed boss Neil Redfearn enough to earn a deal.
Kelly will go straight into the squad for Saturday's clash at Leeds and looks set to play central defence.
“He played in a central role in the behind closed door games and that’s where we will probably utilise him to begin with, but he is very versatile.
“I said from the beginning that our lads might need some help, and these are the type of players that will come in and help.
“What we are bringing in now are players who will make a difference, they will give the boys a lift during an important time when we need that boost.”