The South would not have seceded if northerners had not resorted to violence (Harper's Ferry), cheering on/celebrating those who committed violence (all across the north on the day of Brown's hanging), protected perpetrators of violence (Republican governors of Iowa and Ohio shielded Harper's Ferry perps from prosecution) and promised to commit more violence once Republicans headed the Federal government (Lincoln's political ally Long John Wentworth, editor of the Chicago Democrat promised that northerners were sneaking into the south to assist the next slave uprising).Does anybody truly believe the south would have still seceded if there was no slavery issue? I'm not trying to say there weren't other contributing factors but slavery was the elephant in the room.
Exhibit A evidence for this is: US history between 1789 and October 1859. No antislavery violence, no secession. (The only exception was 1856, the year John Brown did his antislavery terrorist bit in Kansas, and even then, enough northerners were disavowing Brown's terrorism that very few southerners were urging secession.)
But your basic premise is true. If there had been no slavery in the US in 1860 and Lincoln got elected, there would most likely have been no sizeable secession movement. When and how we had magically/counterfactually achieved a slave-free United States is important, however.
If no Africans had ever been brought to America in 1619 or thereafter, there probably would have been no civil war in 1860, but then again you'd have a very white country in that instance and who knows how a "whites only America" would have acted between 1789 and today.
If someone had suggested in 1789 that the new constitution simply outlaw slavery, there would have been no union. Maybe New England and Pennsylvania could have formed a union on such a basis, but the union as we know it would have been broken up before it started.