You are right: you cannot prevent a determined local admin to get access to this database.
Here's what I -- as a determined user -- would do (I have done it in the past with other programs): I would attach a debugger to the running process and look at the memory to find the plain connection string (of course I am simplifying -- but it works).
Another option would be to use the debugger to hijack your IDbCommand objects to execute arbitrary SQL code, no need for me to find the connection credentials, your application manages the connection for me.
So it really comes down to: how competent and determined are your users? how important is your database? how determined are you?
It's the classical hacker trade-off: how much effort do you want to put into this to discourage your users from cracking your app, knowing that in the end it's always technically possible to crack it?
The bare minimum I would do:
Encrypt the raw connection string. You have covered this part.
Maybe do the encryption in code rather than rely on the built-in app.config section encryption. The problem with the built-in one is that it's self-documenting. People with .net knowledge will know just by reading it how it's encrypted. The security is provided by securing the keys, but they can't really be hidden from a local administrator. There's even a command-line tool to decrypt the section (if you pass the correct keys)! Performing the decryption in code makes it less obvious which algorithm you're using and needs more work to actually decrypt it.
NOTE: you should still use a strong cryptography provider such as those you've mentionned. I am not saying that you should invent your own cryptographic cipher, rather than you may hide the cipher you use in the program itself rather than the config file.
Obfuscate the compiled assembly. Otherwise it's just to easy to open it up in a tool such as IlSpy, and even modify it (e.g. to dump the plain connection string).
Add some debugger traps to prevent attaching a debugger to your process.
Note that both 3. and 4. would be more efficient with a mixed mode assembly (one that contains native code as well as managed code).
This is a start. Maybe other people will have more ideas.
Of course this was just the application side. The embedded database should be appropriately protected as well. It should be encrypted to start with.