Why isn't Remoting Disabled by Default on Windows Server?
I was recently involved in a brief and quite lively Twitter discussion with Don Jones and Jeffery Snover about PowerShell Remoting and why it is enabled by default. I have been involved in a number of discussions about this topic, but never with such a distinguished crowd such as this one. My opinion, and my original comments, where along of the line of “I believe Remoting should be off by default”, “Well, RDP is disabled by default, why not Remoting”, and “SSH has been off by default for years”, whilst the counter arguments were of the form “Because Nano Server” or “You could always customise your environment to be off by default”.
Don Jones posted a follow up to this discussion on PowerShell.org titled “Why is Remoting Enabled by Default on Windows Server?” and asked me to put together a post on why I felt it should be off by default. This was difficult for me to put together, so here goes!
It has long been an industry practice, to disable/stop services which are not in use on your clients and servers. The argument is quite simple, enabled services are vulnerable servers, they expose your devices to potential risks. Simply having Remoting off, unless explicitly required, will reduce the attack surface area and increase the security of our systems.
Even Microsoft has followed an off by default methodology for the past 10 to 15 years. Services like POP3 and IMAP are off by default in Exchange, SQL servers do not listen for IP addresses by default, we need to install roles and features individually. Microsoft learnt from a number of major security blunders in the early days (Code Red, Slammer and even Blaster), and focused on a more secure development and deployment model. Why should there be an exception to this posture that has worked extremely well since Windows 2003?
Linux administrators, and developers of Linux distributions have been in a similar situation in the past. For a significantly long period of time, SSHD has been off by default, and administrators have still be able to manage their server fleets. One of the early reasons for an off by default approach in Linux, was that it ensured that administrators were aware of the risks prior to enabling SSHD. Now it can be argued, that this has been a failure, and I think most would agree. I do, however, believe that the failure is not in the off by default configuration, but is in the lack of documentation covering the secure configuration of SSHD. People in glass houses shouldn’t throw stones, as Remoting can be just as poorly deployed.
Remote Desktop is a great example where Microsoft followed these methodologies. RDP is off for a number of reasons with security being only one of them. Ironically, one of the obvious reasons to have RDP off by default is to encourage the move from on server management to remote management. Whilst adoption has not been as high as was expected (due to issues with third party vendors, administrators and to a big extent Microsoft), it is clearly a sign of how ahead of the curve Microsoft has been.
It has become increasingly dangerous to expose management services, be they SSH or RDP on the Internet. If you have ever been responsible to auditing the log files of a server where SSH or RDP is exposed to the Internet, you will be well aware of the automated scan attempts that are performed. Brian Kreb’s has posted on Internet criminals selling access to Linux and Windows servers whose credentials they have brute forced. What happens when the criminals discover Remoting? Bruteforcing credentials via Remoting should be even easier and have written about just such a thing on previous occasions. Should we be enabling these criminals and providing them with even more machines that they can take over?
Well, we are doing this to an extent right now. Users, administrators and developers have all been busy provisioning virtual machines on platforms like Azure and AWS, and whilst in many cases RDP endpoints are on random high ports, the same cannot be said for Remoting. Those who deployed and manage these systems may be well unaware of the risks that they have introduced to their networks. Moving to an off by default model could protect these environments from this sort of configuration error.
As a side note, it is still interesting to me how Microsoft changed Remoting from off to on by default in Windows Server 2012, with very little fanfare. In 2014 when I presented on Lateral Movement with PowerShell, audiences typically responded with a significant amount of surprise, be they from an administration or security background.
In Don’s post, he talks about the fact we could easily create an off by default environment if we so wanted. I really have to disagree with him, and say that he has missed the point to a degree. Whilst it is true, that we could use a customised gold/master image, Group Policy or some other tool to create an environment where Remoting is off by default, it must be highlighted that the inverse, an on by default environment would be just as simple to create with these tools. If you want it on, then turn it on, it isn’t that hard.
Don also talks about the fact that Remoting is an incredibly controllable, HTTP-based protocol. This introduces the other issue I have with Remoting. Unless you are deploying an Azure Virtual Machine, post install, you will be exposing Remoting over HTTP and not HTTPS. Is this 2015 or 2001? Do we really still need to talk about the virtues of HTTPS? It would be trivial for Microsoft to change the default from HTTP to HTTPS in a manner similar to RDP.
Now let’s talk about the big elephant in the room, or should I say Nano elephant in the room? What about Nano Server?!?!? Nano Server, whilst it is a new concept for some of us, isn’t a completely new in our industry. Whilst I agree, it is probably easier to have Remoting (and WMI) enabled by default, it isn’t like the deployment of a Nano Server is currently a simple process. Currently Nano Server is coming as a standalone WIM image, we need to manually add packages providing roles, and we currently need to join a domain during installation. How hard would it be to have a step enabling Remoting? It is trivial.
Having said all of that, perhaps the best middle ground would be to have Remoting enabled on Nano Server, and off for Core and Full installs? Administrators have more option on the latter two than the former. Perhaps a compromise is in order?
Another side note, why doesn’t Microsoft want to enable Remoting on Clients? If Remoting is safe for Internet exposed servers, shouldn’t it be ok for Windows Clients?
So in summary, why should Remoting be off by default?
- Off by default is an industry standard practice.
- Off by default has been Microsoft practice for over 10 years.
- Linux administrators deal with SSHD off, so can we!
- RDP has been off by default, we lived with that.
- RDP and SSH are actively brute forced, why open up another attack vector?
- Off by default reduces administrative misconfiguration/insecure configuration
- It is just as easy to switch it on, as it is to switch it off.
- Nano Server isn’t as much of a challenge as we thing, but it could be the exception.
As Don said, whether you agree or not, it is entirely up to you and you are welcome to add your polite, professional comments to this post, or over at the PowerShell.org forums where I have cross-posted. Like Don, I wanted to explain and attempt to justify why I think Microsoft’s approach is not correct. I often believe the discussion is more important than the outcome, and I believe this is definitely the case here.
Kieran Jacobsen
MS15-034 Update
I just wanted to let everyone know that over the past few days I updated my MS15-034 code to support HTTPS connections. The work involved was much easier than I expected, so I felt that it was worth including.
Working with HTTPS is pretty simple. I have followed the usual convention and defined the –UseSSL parameter, it should be noted you will need to specify a port with the –Port parameter as well. Typically –Port 443 –UseSSL will perform what you need.
Let’s take a look at a few quick examples.
1) Testing a Windows 2012 server with HTTPS and determining if it is vulnerable:
2) Invoking the DOS, this time there is a custom port number in use:
One thing to note, the certificate will be validated, so make sure it is trusted/valid etc.
I am still seeing and hearing of this attack occurring, with a significant number of systems still remaining unpatched. I still haven’t seen any code examples supporting Remote Code Execution (RCE), but I am sure someone has figured that one out and is keeping it very secret.
You can find the updated code at the GitHub repository MS15034, or download the code as a zip file.
Kieran Jacobsen
Exploiting MS15-034 with PowerShell
Updates have been made available to this code to support SSL. See the update here.
On Thursday morning, I woke up to an extremely busy Twitter stream; the topic which that was on everyone’s’ lips was Microsoft Security Bulletin MS15-034.
MS15-034 is a critical security bulletin impacting HTTP.SYS, which forms a core component of IIS and a number of other Windows roles and features. The vulnerability described in the bulletin is a Remote Code Execution (RCE) however at the time of the publication of this post, only a Denial of Service (DOS) of the system has been achieved. There are a number of claimed RCE pieces of code on sale, yet none have been verified.
Over the days since the original release of the bulletin and its associated fixes, things have moved quite quickly. Simple detection (more) and exploitation code had been developed, as well as more complex pieces and even a Metasploit module. Now it is fine for those of us who have Linux systems, or even maybe those who have Windows ports of perl, python, curl or wget installed to use a number of these scripts/examples that are out there, however I feel it is important that this be presented in a way that is accessible and understandable by the average Windows administrator.
Chris Campbell (@obscuresec), put together some PowerShell code that would allow administrators to determine if a system was vulnerable. Unfortunately, there were some issues with this code that means it isn’t as effective as it could be. Firstly, Chris’ code doesn’t report if any other HTTP errors are generated, for example, if you specify an invalid page, the code wouldn’t alert you to the HTTP 404 that would be returned. The next issue is that a non-vulnerable system returns a HTTP 400 after the patch, something the code doesn’t pick up on. Finally, the range header values specify do not match what has been specified in other pieces of code. I felt we needed not only PowerShell code to test if a server was vulnerable, but that it would be interesting to see if we could also exploit a vulnerable server.
Now it should be noted, that you simply can't use the .Net WebRequest class to specify the appropriate “Range” header values, you need to use the TCPClient class and run at a lower level. This is more of a separate discussion I will leave for a later post.
Over the course of past few days, I developed PowerShell code, heavily inspired by the Metasplout module, to allow for the testing and exploitation of MS15-034. This code is contained in the module, MS15034.psm1 up on the Posh Security GitHub. There are two functions which will be of interest; Test-MS15034, which allows for the testing of servers, and Invoke-MS15034DOS which is capable of performing a denial of service against a specified target.
Testing
The CMDLet Test-MS15034 requires the specification of a computer name or IP address, and optionally a port number, and then 3 different parameter sets:
- Specifying the -Windows2008 parameter
- Specifying the -Windows2012 parameter
- Specifying a custom HTTP path with the -ServerPath parameter
The first two are simply to simply the testing against lab environments, as they will connect to the URL of the format http://<Computer>/welcome.png and http://<Computer>/IIS-85.png respectively. The third option is to specify your own HTTP path, which is something you are more likely to do in a real world scenario. You can specify any file or structure using -ServerPath, examples could be:
- /index.html
- /CompanyLogo.png
- /images/logo.jpg
Let’s look at some examples (these may not display correctly via RSS)!
1) Testing a Windows 2008 server and determining that it is vulnerable:
2) Testing a Windows 2012 server and determining that it is vulnerable:
3) Testing a Windows 2012 server and determining that it is not vulnerable :
4) Testing a server using a custom server path and determining that it is vulnerable:
5) Testing a server and specifying the wrong operating system (or the default files do not exist):
6) Testing a server using a custom server path, which doesn't exist:
This CMDLet will connect to the URL determined by the parameters, specifying the header Range: bytes=0-18446744073709551615. If the response from the server is a HTTP 416, then it is vulnerable, if the response is HTTP 400, then the server is not vulnerable. Other errors will be displayed and managed accordingly.
Exploitation
Now let’s take an attack to the next level and take the server down! The CMDLet Invoke-MS15034DOS will do the trick for us. This CMDLet works much like the Metasploit module. The CMDLet accepts the same parameters as Test-MS15034, however it will begin by testing if the server is vulnerable, and if so, will then perform a denial of service. The denial of service will be performed by specifying the header Range: bytes=0-18446744073709551615.
In the examples that follow, I am simply using the default out-of-the-box images as the addresses, much like in the previous tests.
1) Invoking a denial of service against a Windows 2012 (R2) server:
2) Invoking a denial of service against a Windows 2008 (R2) server:
3) What happens if you try to invoke a denial of service against a patched server:
Next is a video of Invoke-MS15034DOS against an unpatched Windows 2012 R2 server. Note how quickly the server is taken down. I think it makes a nice alternative to Restart-Computer.
It should be noted that my code will only support HTTP and not HTTPS. I could develop support for HTTPS, but I suspect that in most environments, testing if a server is vulnerable via HTTP will be sufficient.
You can download my module from my GitHub repository MS15034, a zip file can be found here.
I really hope people will find this code useful. I will be putting together some follow up posts on this topic, including some observations around the issue and how I developed the code.
Kieran Jacobsen
Enabling Mobile Device Management with Office 365
Microsoft recently announced that they would be including a Mobile Device Management (MDM) platform as part of Office 365. What this means is that organisations, both small and large now have an extremely easy and powerful MDM available to them, without any additional charge to their Office 365 licencing. Administrators can manage Android, iOS and Windows Phone devices, and enforce various corporate policies and standards.
To support the new MDM functionality, you will need to create two new DNS records in each of your Office 365 domains. I have updated the Posh-Office365CloudFlare script to support the creation of these two additional records. You can create these records via the -MDMEnable parameter.
The process for creating the entries is as simple as:
Register-Office365.ps1 -CloudFlareApiToken <token> -CloudFlareEmailAddress <email> -Domain <domain> -MDMEnable
Just a quick note, I am yet to fully test out the new MDM functionality as none of my existing tenants have enabled for it yet.
Kieran Jacobsen
Automating Office 365 deployments in CloudFlare
A few weeks ago, I wrote about Posh-CloudFlare, a PowerShell module I created for managing CloudFlare hosted domains. Since then, I was working on extending an Office 365 deployment, and realized that what was needed was a script which could automate the configuration of new domains. With that in mind, I developed a new PowerShell script, Posh-Office365CloudFlare.
Let's understand the process for the addition and configuration of a new domain for Office 365.
The process starts with the Office 365 Portal. We navigate to the Domains section, click the "Add Domain" button, and after ignoring the introduction, we proceed to step 1. This step starts with us entering our domain name, let's use our old favorite contoso.com. Now we will be asked to verify that we own this domain, either through the creation of a TXT record or an MX record. The typical method is to use is that of a TXT record, created at the root of our desired domain with a value something like "MS=mx********".
After we create the domain, and the wizard successfully sees the appropriate record, we will be allowed to proceed to the next step. Step 2 isn't one that I usually make use of. I typically don't want to modify my users email domains, nor do I want to add new users at this time. I skip this step and move straight on to step 3.
Step 3 starts with another quick introduction screen, and then we will be asked if we would like the DNS for this domain to be managed by Microsoft. Obviously, we are going to answer no and move on. Finally, we reach an important step, we are asked what we want to do with this domain. First, "Outlook for email, calendar, and contacts", or in other words, email; the second, "Lync for instant messaging and online meetings", which is kind of obvious.
If you select “Outlook for email, calendar, and contacts”, then we will be told to create the following records in contoso.com:
- MX - @.contoso.com - which points to contoso-com.mail.protection.outlook.com (priority 0)
- CNAME – autodiscover.contoso.com – which points to autodiscover.outlook.com
- CNAME – msoid.contoso.com – which points to clientconfig.microsoftonline-p.net
- TXT – @.contoso.com – which contains a SPF record
If you select “Lync for instant messaging and online meetings”, then we will need to create the following records for contoso.com:
- CNAME - sip.contoso.com - which points to sipdir.online.lync.com
- CNAME - lyncdiscover.contoso.com - which points to webdir.online.lync.com
- CNAME – msoid.contoso.com – which points to clientconfig.microsoftonline-p.net
- SRV - _sip._tls.contoso.com - with its appropriate port, weight, priority and target
- SRV - _sipfederationtls._tcp.contoso.com - with its appropriate port, weight, priority and target
Reviewing this list of records, we will notice that the only record that changes for each domain is the MX record. The record consists of the domain name we want to add, with dashes replacing the original dots in the domain name. As you can see in the above example, cotoso.com's MX record points to contoso-com.mail.protection.outlook.com, where as awesomecompany.net would point to awesomecompany-net.mail.protection.com.
What about some records that could actually help our users? What if I said we could redirect sub domains of our own to the Outlook Web Access page? Wouldn't it be awesome if a user entered https://mail.contoso.com into their browser, and ended up with the Outlook Web Access? This can be achieved by creating a CNAME record that points to mail.office.com. Let's have our script create entries for mail and webmail perform this redirection.
Now back to the script.
This was a simple script, it doesn't have any complex logic, it will need the following information:
- CloudFlare API Token and email address; this is obvious as we need to talk to the CloudFlare Client API.
- The domain name.
- Do we want to create mail records? Lync records or both?
This is a very, very simple script, we just need to have a set of New-CFDNSRecord calls, with various controls depending on what we require.
For example, creating the MX record is as simple as:
This script only took an hour or so for testing and development time, however there was quite a bit of effort directed to changes in the Posh-CloudFlare and the New-CFDNSRecord CMDLet. If you look at the diff's between the last few versions, you will notice the following changes:
- The CMDLet now accepts input from the pipeline (in this case via property name).
- Restructure the CMDLet into Begin/Process/End (required for proper handling of pipeline input).
- Implementation of parameter sets.
- Cleanup of the validation of parameters.
I added parameter sets to New-CFDNSRecord with the aim to remove the somewhat faulty validation that I had previously. Whilst this sounded, and looked like it was simple, it actually took a few tried to ensure that the CMDLet would function appropriately. This was really interesting and deserves its own post in the future.
Parameter validation was updated in all of the CMDLets to improve email address validation. Previously, validation consisted of testing for an "@" character. Now I am using a regular expression.
Finally, I have spent some time cleaning up the code, not just within New-CFDNSRecord, but across all of the CMDLets. I have been trying, where possible to use ISE Steroids to ensure that everything I right is neat and presentable; it is a fantastic resource.
My final thought on all of this journey is, why couldn't Microsoft have implemented something like this? Microsoft has integrated the process with a bunch of other DNS providers, including the likes of GoDaddy, Network Solutions, 1 and 1 and even Yahoo Small Business. Why can't it also look at CloudFlare?
You can find the finished script over at GitHub, at Posh-Office365CloudFlare, the script is called Register-Office365.ps1. I have included comment based help with examples.
Kieran Jacobsen
Crossing the PowerShell streams
I was recently working with the PowerCat code which was capturing the Pipeline and Error output of code, and wondered, could I also capture the other messages being displayed? Why couldn’t I also redirect the warnings or the verbose output as well?
Let’s revisit how the redirection of the streams works in PowerShell, and hopefully learn about how PowerShell does things under the covers.
To explore the output streams, I am going to use the CMDLet I have below, which is a varation of the one June Blender used in her post on this topic.
It should also be noted that redirection to a variable is similar as to a file. In my examples I am redirecting to a variable.
So what happens if we run this CMDLet normally? We should see four distinct messages, first is out the text we are displaying by just returning a string, next we will have our write-output, then we have write-warning, and finally we will have the wrote-error message (and its associated trace).
What happens if we decide to assign the output to a variable called $output? Let’s take a look.
As you can see, after running the function, we still saw the warning and error messages. What was stored in the variable? Well if we look at that we can see that the text return and the write-output were successfully assigned to the variable $output.
Ok, so how about we apply the old redirection rules that have existed in every shell since the 80s?
Well this is much as we expected, but wait, the warning was displayed and not captured in the variable. The variable has our two output strings and it also has the write-error message as well. As a side note I love how the when calling $output PowerShell still displays the error in red.
If you are running PowerShell 2.0 (or less), this is the end of the line for you. There is no help for you.
If you are running PowerShell 3.0 (or greater), then how do we capture that warning message? What about the verbose?
With the introduction of PowerShell 3.0, Microsoft included support for capturing the other streams, and they did so in a method which is simple, clean and logical. Microsoft simply extended the stream redirection along the well-known and practiced methods.
Let’s take a look at the streams in PowerShell 3.0:
| Stream Number | Stream Description | Redirection |
|---|---|---|
| 1 | Pipeline/Output/Success | > |
| 2 | Error | 2>&1 |
| 3 | Warning | 3>&1 |
| 4 | Verbose | 4>&1 |
| 5 | Debug | 5>&1 |
| * | All | *>&1 |
Let’s take a look at the previous example again, however this time we will redirect the output of stream 3, warning, to stream 1.
As you can see, we captured this information just as expected.
Let’s try verbose output now.
We can see that simply using 4>&1 captures the output. We can also capture all output using the wildcard, *>&1.
I can hear one person asking, “But what about write-host?”
There are many reasons why you should not use write-host, firstly, Jeffrey Snover, the creator of PowerShell says not to, Don Jones, a PowerShell MVP says “Write-host kills puppies” <link>, however both agree, as does the PowerShell community, that there are some times when it might be handy to use Write-Host. Write-Host is black PowerShell magic, it’s the dark side of PowerShell. It is not something you should do without very thoroughly thinking about what you are about to do.
The only legitimate time to use Write-Host, is when you do not want to interrupt or pollute your stream. If you have a situation where you wish to display text to the user in such a way that it will not be caught up in the pipeline.
I only use Write-Host in one piece of code. I use it as part of my modular alerting framework, where I know that the message needs to be seen by the actual user of the code. I also didn’t want event/alert messages contaminating any of my other pieces of code. When I wrote the code, I knew it was a risky piece of code, however it has a clear cut function and purpose.
So what happens to Write-Host and redirected streams? To show you, I have added a Write-Host line to our previous CMDLet as show below:
Let’s run the code, redirecting all input (*>&1) into $output.
As you can see, the write-host was still displayed, everything else was captured in $output.
So what information is around on stream redirection? There are three good resources, firstly, check out June’s post on Hey Scripting Guy, secondly, PS> Get-Help about_Redirection and finally, there is the Microsoft Connect entry where the functionality was requested.
Kieran
PS. No streams were crossed during the creation of this post.
Hacking with a rubber duck
On the weekend I had the pleasure to present at CrikeyCon 2015. I want to thank everyone involved including the organizers; the other speakers; our wonderful MC, Patrick Gray from Risky Business; and of course the attendees!
This year I chose something a bit different to present on, the Hak5 Rubber Ducky. I started with two (and one failed,) demonstrations in the morning before setting up in the events area to show off some more advanced demonstrations.
As promised, I am posting up my content for everyone to make use of it.
Firstly, the PowerPoint slides can be downloaded here, or viewed on SlideShare here (and below).
I have setup a separate page on this side, Rubber Ducky, where you can find the scripts/payloads and a description with each.
There are a number of links which I found to be extremely useful.
If you have any questions, comments, or feedback please feel free to leave a comment, contact me via this site or send a message to me on Twitter.
Kieran
Revisiting Syslog in PowerShell
I have performed a number of updates to the PowerShell SYSLOG module since this post. You can read the latest post. The module has been renamed to Posh-SYSLOG.
The GitHub location has been moved to https://github.com/poshsecurity/Posh-SYSLOG.
The module is now available on the PowerShell Gallery.
I often wonder, as I am sure most developers do, if people ever actually read and use the code that I post online. Was it helpful to them or was it useless? Did they use it for something interesting? One piece of code which I know people do use, is my PowerShell SYSLOG code.
A few weeks ago, a user opened my very first GitHub issue! This issue appeared at first to be simple, but as the user and I started to delve into the complexities of the various SYSLOG RFCs, I realized it was far from it.
Before we get into the issue, the code and the resolution, it is worth highlighting that there quite a few IETF RFCs that relate to SYSLOG messages. The two primary ones being:
- RFC 3164 - BSD SYSLOG. This actually wasn't an IETF standard.
- RFC 5424 - IETF SYSLOG. This is a IETF standard. This obsoletes RFC 3164.
There are also RFCs:
- RFC 3195 - Reliable Delivery for SYSLOG
- RFC 5425 - TLS Transport Mapping for
- RFC 5426 - Transmission of SYSLOG Messages over UDP
- RFC 5427 - Textual Conventions for Syslog Management
- RFC 5848 - Signed Syslog Messages
- RFC 6012 - Datagram Transport Layer Security (DTLS) Transport Mapping for SYSLOG
- RFC 6587 - Transmission of SYSLOG Messages over TCP
I want to send a very big thanks out to the user, DFCH for reporting the issue, helping me understand the RFCs in question and also testing the resulting code.
The Issue
So what was the issue? As DFCH stated:
“The Cmdlet send-syslog.ps1 states in its description to send a syslog message as defined in RFC 5424. However the generated timestamp in the Cmdlet incorrectly formats a timestamp when none is specified by the caller, nor does it validate or convert the timestamp if specified by the caller.”
I will admit that I hadn't ready RFC 5424 or RFC 3164 in a huge amount of detail. As soon as I did it was very obvious that the code was not producing an appropriate timestamp, it also become evident that the overall message I was sending did not meet the RFC specification.
From my analysis, it appeared that I had crossed parts of both RFC 5424 and RFC 3164, ending up with code that wasn't fully complaint to either, and in the long run, not entirely useful.
As DFCH reported, the code didn't not generate the appropriate timestamp, with issues in how it was formatted as well as the precision. Resolving these issues was quite simple, the timestamp could be formatted as recommended by DFCH, this not only resolved the format issue but also increased the precision. Validation of the caller specified timestamp was also easy to implement. I simply changed the parameter to take an object of type DateTime instead of a String.
But this was just the start of the fixes, as I continued to read and understand the RFCs, I realized my messages were incorrectly formatted as well.
Message Formats
There are two valid SYSLOG message structures as defined in RFC 3164 and 5424.
Firstly, RFC 3164 specifies the message structure to be the following:
<PRI>TIMESTAMP HOSTNAME TAG CONTENT
Where:
- PRI - Value based on severity and facility
- TIMESTAMP - What date and time with format MMM dd HH:mm:ss
- HOSTNAME - Who is sending the message
- TAG - Name of the process or program generating the message
- CONTENT - Obviously the message being sent
Next, RFC 5424 specifies the message structure as:
<PRI>VERSION TIMESTAMP HOSTNAME APPNAME PROCID MSGID STRUCTUREDDATA [CONTENT]
Where:
- PRI - Value based on severity and facility
- VERSION - Version of the SYSLOG message (typically 1)
- TIMESTAMP - What date and time with format: yyyy-MM-ddtHH:mm:ss.ffffffzzz
- HOSTNAME - Who is sending the message
- APPNAME - Name of the process or program generating the message
- PROCID - Process ID of the application or script
- MSGID - An Identifier to assist in troubleshooting
- STRUCTUREDDATA - RFC 5424 specifies a method of sending key/value pairs
- CONTENT - Obviously the content of the message
One thing to note with RFC 5424 is that the majority of the fields are optional, you still need to send something to ensure the correct layout however, so the nil value "-" is sent. I should also point out that the RFC states that the CONTENT section at the end is completely optional. If nothing is sent, you don't need to even send the nil value.
My original code on the other hands, was sending messages with the structure of:
<PRI>TIMESTAMP HOSTNAME CONTENT
Where:
- PRI - Value based on severity and facility
- TIMESTAMP - What date and time with format: yyyy:MM:dd:-HH:mm:ss zzz
- HOSTNAME - Who is sending the message
- CONTENT - Obviously the message being sent
How did this happen?
Well, there are a few reasons why this occurred, in no particular order.
- I borrowed some of the logic and ideas from other .Net and PowerShell code samples
- I didn't read either RFC
- The SYSLOG servers I tested against were not stringent in their rendering of messages received.
- I referred to Wikipedia when I was checking that my messages were correctly formatted.
In hindsight, the biggest mistakes were using Wikipedia as my guide, and not testing against a more RFC compliant server.
Using Wikipedia as a source for developing compliant code is probably a bad idea, on this occasion it was a great learning experience. Previously the SYSLOG Wikipedia article did not correctly describe the layout and formatting of the full message, crucially missing out the information about the TAG field. The article has been updated since then with corrections ensuring that it is more understandable. It should be noted however that overall, the Wikipedia article is still focused on RFC 3164 and not 5424.
Let’s look at how we clean-up the code.
Additional Parameters
To ensure that we have enough information to support RFC 5424, I needed to add some additional Parameters (which are not mandatory). These include ApplicationName, ProcessID, MessageID, StructuredData and a switch RFC 3164.
The switch RFC 3164 will simply tell the code to send a message in the RFC 3164 format, instead of sending it via RFC 5424 which is its default.
Hostname Generation
Previously, if the hostname parameter was not specified in my code, I simply used the hostname.exe. Whilst there isn't any problems with this, however RFC 5424 actually specifies in some detail how the hostname field should be determined:
- The FQDN of the server
- A static IP address
- The hostname of the server (Windows will always have on of these)
- Dynamic IP address
- A NILVALUE (-)
I have updated my code to generate the hostname component of the SYSLOG message via the first 3 steps.
Application Name
The Application name can be a little difficult. Typically from within a function we can determine the name of the script which is calling the function via 2 properties of the $myInvocation variable: ScriptName and PSCommandPath. I have used ScriptName with success in the past, and hence decided to use it again. There is one thing to note, if I am sitting at a console and call send-syslogmessage, then ScriptName will be null, and if that is the case, we will simply use “PowerShell”.
Process ID
The Process ID is new requirement to ensure RFC 5424. We get this simply from the $PID global variable.
Message ID and Structured Data
These two will always be user specified, if the user doesn't specify them, then send the default RFC 5424 nil value of "-".
Message Generation
Now that we have all of the information required for either RFC, we can now look at message generation. When it comes time, I simply have an ‘if’ statement that controls which format we want to use. The script will then format the timestamp and message accordingly.
Message generation looks like the following:
For RFC 3164, I fixed up the timestamp, and also added in the application name. For RFC 5424 there are some significant changes. I now correctly include the SYSLOG version (1), and then included the corrected timestamp, application name, process ID, message ID and structured data.
The Future
If there was a demand, I would be interested in extending the CMDLet to support the transmission of messages via TCP, as well as sending signed messages. Right now, I don't have a need for such things.
Conclusion
Now that all of those changes have been completed and tested, I have pushed the changes up to the PowerShellSyslog GitHub repository.
I want to thank DFCH again for raising the issue and helping me through the development of the fixes.
Kieran Jacobsen