Derby-born Smith originally left the Minstermen following their relegation to the Conference in May 2004 after completing more than 80 senior appearances at Bootham Crescent.
The 29-year-old has since had spells with Stafford Rangers and Worcester City before playing under York's new manager Gary Mills at Tamworth. He agreed a one-year contract with the Stags in July and has already made a dozen appearances for them.
Smith could go straight into the squad to face Kidderminster Harriers in the FA Cup fourth qualifying round at Aggborough Stadium this weekend.
"I'm sure he'll do well for us and I'm delighted to have him here," Mills told BBC Radio York. "He was my skipper at Tamworth for last season so I know all about Chris.
"I'm delighted to have him in to be honest," he added. "He's become available which surprised me. He's done superb at Mansfield. I've watched him a few times and I know how well he's been doing.
"Things have changed at Mansfield. They've had a big influx of money. They've
decided to get some more centre-halves in and he's become available."
Earlier this week Mills invited back released former winger David McDermott to the club, but has otherwise warned of imminent depatures in what he considers a much oversized squad.