If you wish to install WordPress on your sub-domain, there will be changes on the Domain section as per the screenshot below:

1. Login tocPanel via yourdomain.com/cpanel.
2. Locate Softaculous Apps Installer.
3. Locate WordPress follow by Install.
4. On the Choose Domain section, choose your desired sub-domain on the drop down menu.

The In Directory section should be blank or else the WordPress shall be install at sub.yourdomain.com/wp