It does depend on how long you want to drive for day trips. We stayed a week in the Aveyron in Marcillac Vallon in 2012, but some of the places we visited eg St Eulalie d'Olt and Estaing, which are both beautiful, would be too far.
I agree with Ken about Conques. It is special and the Tresor is wonderful - such intricate workmanship. Definitely visit Najac and stop at the belvedere for a view of the village. St Cirq Lapopie is about 38 kms and is a Plus Beau Village. It is quite busy but is beautiful. Even Cordes sur Ciel is only about 45 kms away, and Penne about 55kms. We would do a day trip of that distance, but some people consider that too far. Belcastel is another. I always look at the stations of the cross in churches and the one there has interesting ones with slumped glass features. St Antonin Noble Val ( where the movie The Hundred Foot Journey was filmed ) is a lovely small town, and, of course, Figeac.
It is a lovely area. Have fun.