The Symantec Mail Security for Exchange (MSE) 7.10 update is now available! This release includes a central console that enables administrators to manage multiple servers from one location.
SMSMSE 7.10 (Symantec Mail Security for Microsoft Exchange) is one of the products I use, and I recently needed to setup a few of Edge Role servers with SMSMSE 7.10. After installing SMSMSE on all of the Edge servers, the next step was to setup control from a central location using a single SMSMSE interface – oh boy, was I in for a wild trip.
Before we get started, let’s have a look at
This will be more of a how-to guide for dealing with issues if you ever find yourself in a position similar to mine. This will not be your usual install this, click here – that part is simple; what isn’t simple is getting SMSMSE console to operate over a network without a domain.
This is done using Exchange Server 2019 on Windows Server 2019.
I’ll tell you a bit more about my setup.
Let’s pretend for the sake of this narrative that I have two Exchange Edge servers that are not in a domain but rather in a workgroup on a DMZ network. They only have a DNS connection to DC and CAS servers on the LAN, as well as a few LDAP ports, so I can interact with them by name (important for Exchange functionality).
Because local Administrator accounts on Edge computers have been blocked, I established a custom Administrator account (InfoAdmin). Because there is no domain, it is critical that each computer have the same login, password, and group membership!!
I installed complete SMSMSE 7.10 on both Edge roles and intended to administer them centrally from a single computer – I tried with a client PC in the DMZ and on the LAN.
If you want to administer SMSMSE from your LAN, ensure sure port 8081 is open on your firewall, network/router, and virtual machine.
For this tutorial, I’ll link ExTest1VM (which has SMSMSE installed) to ExTest2VM (which also has installed SMSMSE)
If you want to manage SMSMSE installations from a single client that doesn’t have Exchange and just serves as a console, run the SMSMSE installer and choose Custom install.
Then just install the Server Management Console
Now that we’ve had SMSMSE and our central console set up, we can start adding our first server (ExTest2) to the console.
I choose Assets from the Home menu after logging into the SMSMSE interface.
Then I choose Global Group – Exchange 2019, and then Add server(s)…. Under the Tasks menu
I received an error saying there was no domain… I don’t mind since I can locate my servers by name, despite the fact that they are Edge servers in a DMZ workgroup (check out my Exchange Edge and pFSense DMZ guides)
OK
I’ll type “ExTest2” in the “Server name or IP” field, which is the name of the server I want to join. After that, choose >>.
OK, I’ll go with SMSMSE 7.10 on Exchange 2019 |
No issue, my server has been added to the list of Selected servers. OK
Close the Assets menu now.
Select the Change… option on the Home screen now.
Select ExTest2 from the Global Group – Exchange 2019 | click Select
Now, remember the process above since you will be doing it many times. Removing and then reading the server to ensure that it works after you’ve made a modification.
Okay, I entered my InfoAdmin account (which is the VM’s primary admin account) and password. OK
The request failed with HTTP status 401: Unauthorized, which was the first obstacle we encountered from SMSMSE.
That’s all right.
I was able to finish this one pretty fast. SMSMSE requires IIS to function, therefore it was installed.
We’ll now go to ExTest2 VM (the machine we want to connect to through SMSMSE interface) and open IIS Manager, expand Sites and choose Symantec Mail Security for Microsoft Exchange, and select Authentication from the center screen – you’ll see Anonymous, ASP.NET, and Forms authentication.
There are no Basic, Digest, or Windows Authentication options.
Start Add Roles and Features from Server Manager and choose Digest, Basic, and Windows Authentication from the Server Roles screen under Web Server | Web Server | Security | proceed through the installation – then reboot the system.
Now, on the ExTest2 VM within IIS, we have new authentication choices – go back to the Symantec website in IIS on ExTest2 and choose Authentication. Restart IIS after enabling Windows authentication. After you are unsuccessful with the login, you will need to disable/enable Windows Authentication in the SMSMSE console to trigger the login screen again.
Let’s return to ExTest1 and attempt the SMSMSE console connection to ExTest2 again.
Error once again. This one is a little more – should we say – descriptive. You’re completely oblivious to what’s going on. ExTest2Infoadmin was one of the login choices I tested, and I shortened my password… Nothing seemed to help.
What is irritating about this – apart from the absence of an apparent error notice and nothing in the Windows/Symantec logs – is the lack of documentation that covers this installation situation (if there is, please accept my apologies; I couldn’t locate it).
So, let’s go back to ExTest2 and test a few items one by one.
First, check to see whether the SMSMSE Admins and Viewers groups are present on ExTest2 — they are.
Let’s check whether our InfoAdmin user belongs to the SMSMSE Admins group, which he does. All is OK since the account is also a member of the Administrators group.
My guess was that I was still fighting permissions after an hour or two of reading the internet and official literature for SMSMSE on the Broadcom site and tried different things, so I performed the following.
The SMSMSE Admin and Viewer group has no access to Symantec directories on the system.
So I gave SMSMSE Admins full access to the following folders:
Symantec Symantec Symantec Symantec Symantec Symantec Symantec Symantec Symantec (check if the subfolders got same rights you assigned to the top folder)
SymantecCMaF.c:Program FilesSymantecCMaF.c:Program FilesSymantecCMa
SymantecSMSMSEC:Program FilesSymantecSMSMSE
C:Program DataSymantecSymantecSymantecSymantecSymantec
Symantec Shared C:Program Data
(Program Data is a hidden folder in the system drive’s root.)
I was having trouble giving permissions in the Program Data folder (I also have Symantec Endpoint Protection), so I simply hit Continue…
The problem is that even after this – I was still having issues – I would receive an empty error screen.
Then, in a fit of desperation, I looked through the txt files within the SMSMSE installation and discovered this tiny gem inside the readme file – which I never examine inside any installer. I read the documentation and instructions, but I never read the readme file in an app’s installation directory. In life, I suppose there is a first time for everything.
So, on the ExTest2 server, I launched regedit (as admin) and proceeded to the following keys.
[HKLM]SOFTWAREWow6432NodeSymantecInstalledApps
[HKLM]SOFTWARESymantecSMSMSE7.10Server
Select Permissions from the context menu when you right-click on a value.
Select Apply | OK after giving both SMSMSE Admins and Viewers permissions. This should be done for both registry values.
The server should be restarted.
Okay, I was anxious to connect from ExTest1 to ExTest2 server right now.
Still no luck, maaan. It’s really aggravating.
I also restarted the ExTest1 server and attempted to connect to ExTest2 again. I received the same blank error, and when I clicked Cancel, I received something different. At the very least, there’s a decent error notice.
“You are not permitted to enter. The user is either unauthorized or has restricted access to the server.
As a result, we’re still fighting permits. Exhausting.
Symantec was a little more helpful with this one.
On the ExTest2 server, click search and type dcomcnfg. Run it with administrative privileges.
Expand Component Services | Computers | My Computer | DCOM Config (a notice will appear, choose Yes) in the center screen, locate SMSMSEGUI Class, right-click on it, and pick Properties.
Toggle to the security tab.
Customize was selected for Launch and Activation Permissions | Access Permissions and Configuration Permissions. On each of the three, I clicked Edit and granted the SMSMSE Admins and Viewers full rights.
Repeat the procedure for each of the three edit buttons | both Admins and Viewers have full privileges.
When you’re finished, click OK and shut everything. For good measure, reboot the server.
Return to ExTEst1 to check the connection to ExTest2.
We’ve finally made contact!!!!
What a thrilling opportunity. Let’s make a short modification and deploy changes to make sure everything is working properly.
Make the necessary adjustments.
The SMSMSE service is not running 🙁 Will this sequence of errors ever come to an end?
Returning to the ExTest2 server to check on the SMSMSE service’s status
ExTest2 is used to execute the service.
And it’s set to operate as a local account, which is OK.
If I go to the ExTest2 server’s main page SMSMSE, I can see the following:
The service has begun.
However, if I check the same location on the ExTest1 server, which is now linked to ExTest2, I get the same result.
The current state of affairs is unclear. “Wonderful”
There were no logs or error messages to assist me with this one, and there were no hidden readme files to help me this time.
What I experimented with
These are some of the things I tried to fix the “SMSMSE service is not started” issue, but none of them worked!
I was so desperate that I turned off the firewall on both servers, but it didn’t help.
More permissions in a variety of locations – that didn’t work.
I attempted a small thing within Windows Server 2019 using Powershell after some additional research about SMSMSE (as admin). I attempted to connect to ExTest1’s service from Extest2’s server.
I tested out two different services. SMSMSE service and wuauserv (Windows Update)
Get-Service -ComputerName ExTest2 -Name SMSMSE 
I was able to access wuauserv but not SMSMSE as a consequence of the strange findings.
I then downloaded the Sysinternals Process Explorer program and gave the SMSMSE Admins and Viewers full access to that process – but no luck.
Because to the new concept in Windows Server 2019, you can no longer readily access services on other computers. So I immediately deactivated it on ExTest2 by adding the following to the registry (run in cmd as admin)
HKLMSYSTEMCurrentControlSetControl /v RemoteAccessExemption /t REG DWORD /d 1 /f reg add HKLMSYSTEMCurrentControlSetControl /v RemoteAccessExemption /t REG DWORD /d 1 /f reg add HKLMSYSTEMCur 
I also tried adding the SMSMSE exception to the RemoteAccessCheckExemptionList entry in HKEY LOCAL MACHINESYSTEMCurrentControlSetControlSecurePipeServersSCM on the ExTest2 server.
After that, I restarted ExTest2 VM, and I was even able to access SMSMSE service remotely from ExTest1 server on ExTest2 at one point – however SMSMSE console refused to go away. When connected remotely, the status of the SMSMSE was unknown.
What worked for the SMSMSE service doesn’t function anymore?
I got to the bottom of it after a lot of research and reading, and a few more hours wasted, and the answer was easy.
I activated the Administrator account on ExTest2 VM as a last resort. In the SMSMSE Admins group, I also included the built-in Administrator. My InfoAdmin has the same privileges as the built-in Administrator. Exactly the same (as I can see)
After that, I re-added ExTest2 to the SMSMSE console on ExTest1 and re-provoked the login page (through IIS Windows Authentication disable/enable) with a different user. I used the Administrator user this time. Check whether Windows Authentication in IIS is enabled under the Administrator account if you receive Error 401 while entering the Administrative username. If you don’t get an authentication page like the one below, delete the server from SMSMSE, deactivate Windows Authentication on the server you want to add, and then re-add the server.
I was able to log in and see that the SMSMSE Service had begun!!! What a wonderful day it has been.
Finally, I was able to deploy the modification from Extest1 to Extest2.
What an eventful day.
Conclusion
To say the least, the situation I described above was aggravating. The scenario of SMSMSE deployment on the Exchange Edge role is supported. This situation – when Edge is in DMZ, without a domain and additional computers surrounding it – should be properly handled. You want to harden the Edge job as much as possible since it is more exposed to the internet. On exposed computers, I don’t recommend using the default Administrator account.
Another option is to handle SMSMSE from each computer separately, rather than having a central administration and configuration deployment point.
Disclaimer
