Slavery wasn't what pushed the poor southerners to secede, it was based on economics and the independence of the states.
This is an apologist perspective
I suggest you look at the states secession documents.
First two sentences of Georgia's document: "The people of Georgia having dissolved their political connection with the Government of the United States of America, present to their confederates and the world the causes which have led to the separation. For the last ten years we have had numerous and serious causes of complaint against our non-slave-holding confederate States with reference to the subject of African slavery. "
First two sentences of Mississippi's document: "In the momentous step which our State has taken of dissolving its connection with the government of which we so long formed a part, it is but just that we should declare the prominent reasons which have induced our course.
Our position is thoroughly identified with the institution of slavery-- the greatest material interest of the world. "
Texas: "In all the non-slave-holding States, in violation of that good faith and comity which should exist between entirely distinct nations, the people have formed themselves into a great sectional party, now strong enough in numbers to control the affairs of each of those States, based upon an unnatural feeling of hostility to these Southern States and their beneficent and patriarchal system of African slavery, proclaiming the debasing doctrine of equality of all men, irrespective of race or color-- a doctrine at war with nature, in opposition to the experience of mankind, and in violation of the plainest revelations of Divine Law. They demand the abolition of negro slavery throughout the confederacy, the recognition of political equality between the white and negro races, and avow their determination to press on their crusade against us, so long as a negro slave remains in these States."
South Carolina: "In the present case, that fact is established with certainty. We assert that fourteen of the States have deliberately refused, for years past, to fulfill their constitutional obligations, and we refer to their own Statutes for the proof.
The Constitution of the United States, in its fourth Article, provides as follows: "No person held to service or labor in one State, under the laws thereof, escaping into another, shall, in consequence of any law or regulation therein, be discharged from such service or labor, but shall be delivered up, on claim of the party to whom such service or labor may be due."
This stipulation was so material to the compact, that without it that compact would not have been made. The greater number of the contracting parties held slaves, and they had previously evinced their estimate of the value of such a stipulation by making it a condition in the Ordinance for the government of the territory ceded by Virginia, which now composes the States north of the Ohio River. "
I dunno dude. Seems like it was kind of about slavery.
https://sunsite.utk.edu/civil-war/reasons.html