I have a SharePoint Online Document library that contains only the out-of-the-box “Document Set” content type. Using the Ribbon I create a new Document set and then open it. From the ribbon I then click the “New Folder” button and a new folder is created as expected.
Examining the properties of the new folder I find that SharePoint has assigned the Content Type to be “Document Set” and not “Folder” as I would expect. SharePoint does provide the ability to change the content type to Folder using the Content Type dropdown selector in the Edit properties form.
How can I configure the Library to use “Folder” content type instead of “Document Set” content types when users create new folders?
Can you confirm that you have the Document content type available in the library? This also should be set in the document set settings page as an available content type.