Sending SYSLOG messages to TCP hosts and POSH-SYSLOG V3.0

The Posh-SYSLOG module has been very popular since its first release in 2013. SYLOG provides a common integration point into enterprise monitoring, alerting and security systems and administrators and developers often need to push messages from their scripts and automation activities to a SYSLOG server.

There are two common pieces of feedback, TCP support and improving the performance. I am excited to announce a new version of POSH-SYSLOG, which introduces TCP support and a number of performance improvements.

Kudos must go to Jared Poeppelman (powershellshock) from Microsoft who provided the TCP logic, optimised performance, and added additional Pester cases. I took some time out to make some additional improvements on top of Jared’s work. Due to the significant changes and additional functionality, I am considering this to be version 3.0.

The easiest way to get the module is from the PowerShell Gallery using Install-Module -Name Posh-SYSLOG. You can also clone the GitHub repository.

The first big change by Jared, was to implement the Begin {} Process {} End {} structure for Send-SyslogMessage. With this structure, we can leverage the pipeline to send multiple messages. I have developed scripts where I needed to read application logs and then send them to a SIEM product using SYSLOG; access via the pipeline simplifies these scripts and hopefully improves their performance.

The logic around determining the correct value to be sent as the hostname has been cleaned up and refined. This piece was free of issues, however there were some small tweaks that potentially improved performance. The function is now called as part of the Begin {} block, improving performance for bulk message transmissions. The logic has been moved out to its own internal function, allowing for separate testing and better mocking opportunities in Pester.

Another source of performance improvement is the removal on the need to call Test-NetConnection. This is a dramatic source of improvement when Send-SyslogMessage is executed in a workgroup (that is, a non-domain joined) environment. Previously we called Test-NetConnection to determine the correct network interface that we are using to communicate with the SYSLOG server; now we simply ask the socket for the source IP address and then use Get-NetIPAddress to check if this is a statically assigned address.

All the network functionality and calls have been moved to internal functions. This helped with testing, I can now mock all the network activities which allows for better testing with Pester. The network tests are now much more reliable.

Finally, Jared and I have increased the number of Pester tests. I have tried to aim for over testing everything in the hope that all potential issues are flushed out. With a massive upgrade to the functionality, and such a refactoring have the potential to introducing issues. I am confident that things have been appropriately requested. If issues are found, please raise them via GitHub.

So what is the future now for Posh-SYSLOG? Well for now, I just want to ensure there are no bugs or issues, after that I want to look at implementing the other commonly asked feature; TLS support.

The PowerShell community has been amazing, I have been lucky to have such wonderful community contributions over the past few years. A massive thanks to Jared, Ronald, Xtrahost and Fredruk.

Kieran Jacobsen

How the Skype team failed at PowerShell

The Skype for Business team recently sent out an email to preview customers with guidance around the Skype Meeting Broadcast feature. As I was away, Readify’s CIO, Tatham Oddie was the one who actioned it. The guidance in this email has a number of issues, leading him to post his thoughts up on Twitter, needless to say, he didn't like their advice!

The instructions were pretty simple, as you can see:

A full copy of the email can be found here.

I was shocked when I saw the guidance, from requiring elevation, to changing the execution policy without informing the user, and then a Lync reference to top it all off. I felt that some things needed a discussion.

Unneeded Elevation

The user is asked to run an elevated PowerShell session on two occasions. This is two too many.

For the first instance, the user is asked to run an elevated PowerShell simply to execute Get-Host. GET-HOST! There is no conceivable reason why you would ever need to be elevated to run this CMDLet. I think Get-Host is probably one of the simplest CMDLets in existence, yet the Skype for Business team felt like it needed to be elevated.

Open PowerShell as an administrator (Right click on icon and click run as administrator)

Now the second elevation attempt is almost legitimate. During the step titled “Connect to your Office 365 tenant through remote PowerShell”, we will be executing Set-ExecutionPolicy. Unfortunately, I will discuss later why this command shouldn’t be here, and hence there is no reason for elevation.

Open PowerShell as an administrator (Right click on icon and click run as administrator) and issue the following cmdlets:

Any time you elevate, you need to be more aware of what you are doing. In most cases, elevation is only required if you are making changes to the local system. Instructions to configure a cloud based service requiring elevation should be ring big honking alarm bells, maybe even giant ones. Why on Earth would I be making changes locally, when configuring a cloud service?

I don’t run as local administrator; my every day account is not a member of the local administrators group. This has the unfortunate side-effect of breaking much of Windows 10 (thanks Gabe), however it dramatically improves your security posture. Even if UAC was bypassed, my account isn’t a member of the group, it doesn’t have the privileges you seek. When I need to elevate, it’s a conscious decision on my part, do I want this app to do this, do I want to make this change? I very much believe that all users, be they home users, developers, or system administrators, should run in this model. As a side note, its painfully obvious that most of the Microsoft development team runs with local admin.

Changing The Execution Policy

Set-ExecutionPolicy Unrestricted

I always get a bit upset when someone or something changes the execution policy and in the guidance, that is exactly what the Skype for Business team want to do. Not only have they decided to make this change for me, but they have picked one of the least secure settings, unrestricted. They haven’t even made an attempt to switch it back and the end of the execution process.

The first thing I want to point out is that the Skype for Business CMDLets will work perfectly on execution policy of AllSigned and RemoteSigned. They are after all signed with a code signing certificate issued to Microsoft. After going through all of the effort to get their modules signed, the Skype team simple went YOLO, let’s set the policy to Unrestricted.

They could have of course gotten the user to run simply that PowerShell session as unrestricted, or they could have recorded what the policy was before they changed it, and switched it back. Like this:

There has also been no attempt at even checking what the execution policy was previously, perhaps all would have been OK? Why not have this as a prerequisite step?

Finally, they have made the assumption that the user receiving this guidance can change the policy. I have worked in a few environments where group policy would step in and prevent setting the execution policy to unrestricted.

What is an Admin with MFA to do?

It would have been great to see a mention of what an admin with MFA enabled, should do to run these commands. Thankfully the Skype for Business team allow the use of App Passwords, unlike their poorly informed and aggravatingly annoying peers in the SharePoint and Exchange online teams who reject app passwords.

Is that a Lync reference?

$LyncSession = New-CsOnlineSession -Credential $credential

I know Tatham and I are being pedantic, but a Lync reference, seriously? Microsoft announced that Lync would be replaced by the Skype for Business branding a year ago!

What the email should have looked like

I don’t like tearing things apart without helping to fix them, so how could the Skype team have done things better?

Well here is one that I prepared:

To get started with Meeting Broadcast please follow these steps:

Setup PowerShell
Open PowerShell

To check your PowerShell version, enter this cmdlet:

Get-Host

You must have version 3 or higher. If you have a lower version, install PowerShell version 4: http://www.microsoft.com/en-us/download/details.aspx?id=40855

To check you have the Skype for Business module installed, enter this command:

Get-Module -ListAvailable -Name SkypeOnlineConnector

If there is no results listed, Install the Skype for Business module: http://www.microsoft.com/en-us/download/details.aspx?id=39366

To check you have the correct execution policy level, enter this command:

Get-ExecutionPolicy

If the output is undefined or restricted, please view our guidance on setting the execution policy to RemoteSigned: http://www.microsoft.com/some-guidance-here

Restart your computer (as it may not be prompted to do so).


Connect to your Office 365 tenant through remote PowerShell
Open PowerShell and issue the following cmdlets:

$Credential = get-credential
$Skype4BusinessSession = New-CsOnlineSession -Credential $credential
Import-PSSession $Skype4BusinessSession

Note: If your account has Multi-Factor Authentication enabled, please ensure you use your username and an app password.

As you can see, there are a number of changes.

  • There are no elevation requests in the instructions. We check the prerequisites with no elevation, and the connection instructions no longer require elevation either.
  • There are now appropriate checks to ensure the Skype for Business Online Connector is installed. This is a simple touch, but makes the process much easier for users.
  • Checking what the Execution Policy is, and we link to Microsoft guidance on what the appropriate policy is, and how to change the policy.
  • I removed the Lync reference.
  • Changed the restart wording. Sometimes when you install the module, it doesn’t correctly prompt for a restart, but it does indeed need a restart.
  • Added a note around what to do if MFA is enabled.

None of these changes were very difficult, nor where they very time consuming. I really don’t understand why no one at Microsoft picked up any of these issues prior to the email being sent off. I can’t see any excuses for missing this issues, they were painfully obvious.

Kieran Jacobsen

White-listing Tor in CloudFlare with PowerShell

CloudFlare has been one of those technologies that has dramatically improved the Internet since its arrival. CloudFlare provides a number of amazing features, from DDOS protection, traffic analytics, performance optimizations and even free SSL! To top it all off, CloudFlare off the majority of their features for an amazing price – FREE!

Unfortunately, there are a group of users who have suffered with the rapid adaption of websites being protected by CloudFlare; Tor users. Whilst the majority of us receive the benefits of CloudFlare, it has made life for Tor users extremely miserable.

What is the issue with CloudFlare and Tor?

To understand the issues CloudFlare and Tor, we first need to understand how CloudFlare protects a website. When you visit a protected website, CloudFlare performs a risk assessment of your IP address. If CloudFlare decides that you are friendly, then you get to view the website. If it decides you are suspicious, then you will either be prompted with a CAPTCHA request or, in the extreme cases, it will deny access to the website.

The Tor is an interesting network, developed with the aim to protect privacy and defend against surveillance; the network has become a bit of a hive of scum and villainy. Due to the high use of Tor for malicious activity, CloudFlare is naturally suspicious of any connections coming from a Tor exit node, which results in a high risk assessment for all Tor users accessing protected websites.

A high risk assessment results in lots of CAPTCHA requests for Tor users to complete. Every protected page that a Tor user visits, will result in a CAPTCHA request, whilst most of us don’t mind completing these, consider how you would feel answering one every 15 to 20 minutes. It impacts productivity and your sanity.

What is the solution?

The trust of the matter is that there isn’t a simple solution, there isn’t a magic “trust Tor” button in the CloudFlare. It is down to individual website operators to white-list the Tor exit IP addresses; this isn’t trivial, requiring operators to maintain a white-list as the Tor network changes, adding new exit IP addresses that join the Tor network, whilst removing those that leave.

Now is the best time to point out the risks of white-listing. White-listing isn’t without its own risks! When you white-list an IP address in CloudFlare, you are telling it to ignore how risky the traffic coming from that address might be. Not all Tor users legitimately want to view the content of your website; some Tor users are malicious, they are out for blood. White-listing significantly changes the security protection of your site, and you need to be extremely aware of the risks it introduces.

Set and Forget White-Listing

Donncha O'Cearbhaill developed a Python based script with the aim of providing a “set and forget” mechanism for white-listing Tor exit IPs. Donncha’s Automatic CloudFlare Tor Exit Whitelister takes your CloudFlare API token and email address and does all the hard work. The script will grab the latest listing of the Tor exit IPs, and then updates CloudFlare’s white list as appropriate.

When I saw the script, my immediate reaction was “what an amazing idea”, my next was, “how could this be made more accessible to more website operators?

Turning to PowerShell

The obvious answer to encouraging more web site operators to white-list Tor exit nodes is to provide the process in more accessible formats. For Windows users, the best option is to provide the process as a PowerShell script.

So I sat down, learn some Python, and then worked on developing a PowerShell script with similar functionality as Donncha’s. This was a super fun and interesting process. In the end, I deliberately tried to stick a similar logic flow, similar parameters are supported (via parameter liases), and the scripts default output is similar. I added some additional verbose and debug options and added progress bars for long operations. I haven’t included support for specifying the CloudFlare API token and email address via environment variables.

The final result can be found over at GitHub, at https://github.com/poshsecurity/PowerShell-CloudFlare-Tor-Whitelist. All of the functionality is provided by the Set-CloudFlareWhiteList.ps1 script.

Please feel free to raise pull requests, issues, or provide any feedback that you may have.

Usage

I have included comment based help, with examples on how to make use of the script. There are three common usage methods.

  1. Specifying a CloudFlare token and email. This will white-list exit node IP addresses across all of the domains in your CloudFlare account
  2. Specifying a CloudFlare token, email and a DNS zone. This will white-list exit node IP addresses for a specific DNS zone.
  3. Specify a Token, email and optionally a DNS zone, and the -ClearFlags parameter. Remove all Tor specific rules for all domains in your CloudFlare account, or for a specific DNS Zone.

Finishing Up

I want to thank Donncha for his work in developing the original Python script. Without his work, I wouldn’t have been able to develop mine.

If you want to find out more about CloudFlare, Troy Hunt has put together a terrific PluralSight course, Getting Started with CloudFlare™ Security. He covers the basics including migration, SSL configuration and even HSTS.

Kieran Jacobsen

Why Remoting vs SSH vs RDP Shouldn’t Be A Thing

Microsoft recently announced that they would be introducing SSH support in the coming Windows Server release. It has taken three attempts by the teams within Microsoft to gain the necessary approval to do this, and finally, third time is the charm. Hats off to those within Microsoft who have had the courage to keep to their guns with this one and get SSH included. You truly have made a positive improvement.

The introduction of SSH opens up a world of possibilities for administrators, developers and integrators; unfortunately, there seems to be quite a few in the industry that don’t share in the excitement. 

I think some people misunderstood why people were so excited. Some went so far as accusing a few of us of shouting “Death to Remoting”. I don’t see this as the death of Remoting, it is naïve to think that SSH will replace Remoting; SSH will compliment Remoting and solve a number of challenges which currently exist with Remoting.

Native support for SSH both as a client and as a server opens up a number of functions and tasks which until recently were only achievable via third party utilities, modules and just plain hacky code. Carlos Perez and I have both written PowerShell modules (his is better), that integrate with SSH, Posh-SSH and SSHFunctions respectively. We wrote these to bridge a number of gaps, gaps that will/should be removed with native SSH support. 

This is a new world for administrators and developers alike, be that those who work primarily in the Microsoft space, as well as those working with Linux.

Remoting –ne SSH –ne RDP

It should be painfully clear, but I want to make it absolutely crystal clear; Remoting does not equal SSH, SSH does not equal RDP and RDP does not equal Remoting.

Each of these protocols have a different aim:

  • Remoting (or WinRM) is roughly a remote management protocol
  • SSH provides a Secure Shell for text based management
  • RDP provides remote GUI access for GUI management

We really shouldn’t get these confused, each of them have different goals and aims, and as administrators/developers you need to know each one, when each one is appropriate to use and when each one is inappropriate.

Remoting

Remoting is an implementation of the WS-Management, which is an open, SOAP based protocol for the management of things like servers, workstations, devices and applications. The WS-Management standard was developed by a bunch of smart people at AMD, DELL, Intel, Microsoft and Sun back in 2005.

Unfortunately, WS-Management didn’t really take off, with only 4 major implementations found on Wikipedia: Windows Remote Management (WinRM), an implementation in SUSE Linux Enterprise, a European research project and finally Intel’s Active Management Technology. It is easy to speculate why WS-Management didn’t take off, the most obvious is the heavy overheads of SOAP web services. SOAP was big back in the 2000s, it isn’t as popular now and with good reason.

Believe it or not, I like WS-Management and Remoting. Remoting supports two pieces of software to connect to each other, and efficiently exchange complex pieces of data, allowing administrators, developers and integrators to build rich pieces of software on top. 
Multifactor authentication is pretty limited with Remoting. While you can add two factor into RDP, remoting hasn’t had the same treatment. 

Server authentication in Remoting is typically a complex story. To start with, typical deployments will be based upon HTTP providing limited (read: no) authentication, if you want strong server authentication, then you need to switch to HTTPS. 

The switch to HTTPS for Remoting is where things start to fall apart. A typical Remoting HTTPS deployment will be based upon self-signed or internal PKI. If you stick with self-signed, you either need to manually trust each server’s certificate, or you can skip the CA trust checks. If you want proper certificates, typically you are going need to deploy a CA infrastructure, and then configure each server to use HTTPS; you can’t use Group Policy, it doesn’t support enabling HTTPS for Remoting. Finally, don’t turn off HTTP if you want to keep using the Windows Server Manager. If you want to understand how bad configuring HTTPS for Remoting is, check out the best guide available at Carlos’ Blog, WinRM SSL Certificate Deployment via GPO.

Remoting has unfortunately been missed when it goes for secure configuration guides, the same goes for things like IDS and IPS rules. Whilst there are obvious challenges for IDS/IPS scanning of Remoting traffic that is over HTTPS, thankfully we can apply a lot of the same policies and rules for IIS to Remoting.

I feel a little jaded by Microsoft’s release of a reference implementation of WS-Management for Linux. It should be obvious to Microsoft by now that big companies cannot force the open source community to implement the standards and solutions they want. I was kind of shocked to hear about this one.

Side note: How long till Remoting is RESTful?

SSH

SSH, or Secure Shell, is an encrypted protocol for the initiation of text-based shell sessions on remote machines and was originally developed to replace Telnet, RSH and REXEC. If you ask the average person what SSH is for, they will tell you it is what you manage Linux and Unix with.

Whilst it is possible to use SSH as an integration point into automation and management of remote endpoints, it doesn’t support the exchange of complex object types, it only supports strings. For those who haven’t tried using SSH for large amounts of remote command execution, the closest would be something like invoke-command. Don Jones explained it well, “It’s all text, all the way down to the turtles.” As he goes on to summarise, whilst text isn’t bad for automation, string manipulation isn’t good. I also feel that it is wise to point out that string manipulation is a great way to introduce security vulnerabilities as well.
SSH makes up for all of this, by being extremely secure. SSH supports multifactor authentication with support for both password and key authentication, something that Windows still hasn’t caught up to (natively). 

Server authentication, that is, the authentication of the server by the client (so we know we are connecting to the right server and we are not being MITM), is quite easy. Servers have a public and private key pair, we just need to verify the thumbprint of the public key, and then we can establish a trust relationship (from here to eternity if we so desire). 
Another advantage of SSH is we know extremely well what valid SSH traffic looks like. This is critical in protecting connections either leaving our networks perimeter or those entering from outside. If devices like our IPS and IDS systems know what a valid SSH packet looks like, then we can make an attempt to weed out the nefarious traffic.

Transferring files over SSH is extremely easy, fast and efficient. I have used SSH as a file transfer mechanism in a number of environments, be it at home, in financial institutions or even for backups and server migrations. In banking SSH is an industry standard, much like PGP/GPG for a reason.

In summary, SSH gives us a secure text session in a manner that is extremely easy and without a PKI, unfortunately it is pretty bad at the exchange of complex data.

Side note: Check out Mosh an SSH replacement (not available on Windows yet). I really like the idea’s behind Mosh and can’t wait to see where it goes. I would really love to see a synchronised state console like this one.

RDP

We should all by now, know what RDP is! RDP allows us to connect to remote devices and servers via a graphical session. RDP does remote graphical consoles really well, has a simple but quite reliable security model and supports some limited file transfer options.

RDP is a heavy protocol on a network connection and for obvious reasons. Even with the advances in later RDP protocol versions, sending a graphical interface over the Internet can still be slow and laggy; combine this with file transfers, and you are in for a long wait.
There are a number of ways to add multifactor authentication to RDP, most notably solutions from DuoSec

Server authentication in RDP is actually pretty good. RDP comes by default with a self-signed certificate which does provide a degree of server authentication. You can step it up a notch by moving up to certificates issued from an enterprise CA. Configuration of RDP to use an enterprise CA is extremely simple, you can use group policy all the way. For a great write up on how to set it up, check out Carlos’ post, RDP TLS Certificate Deployment Using GPO.

RDP has been around for a long while, and a significant number of IDS and IPS products have developed rules to detect malicious traffic. Much like SSH, there are plenty of secure configuration guides for RDP from a number of trusted sources.

We need to reduce our reliance on RDP, not just because of how slow it is when managing cloud based infrastructure but also because it doesn’t support automation to any extent. RDP will always exist, there are times when we will need to run a GUI on a remote machine. 

Enter-PSSession

Enter-PSSession deserves a special mention. Providing functionality much like SSH, Enter-PSSEssion allows users to initiate a text based shell session with a remote device all over Remoting. Yes, in many ways it does provide a similar functionality to SSH, but from personal experience it can be a bit slow and clunky. 

Why is Enter-PSSession a bit slow and clunky? Well the best way to understand why, is once again by Don <link>. With Enter-PSSession, keystrokes are not transmitted in real-time, instead, each whole line is transmitted, executed, and then the results sent back to us. With all of the SOAP overheads, you can start to see where things get bogged down.

The right tool for the right job

Different tasks require different tools. You don’t use a sledge hammer to drive a screw, you need a screwdriver.

If you are running a graphical interface directly on a server (for an application that doesn’t support running remotely), then you need RDP. If you have to run the GUI on the box, you don’t have any other option but to run the GUI on the box via an RDP session. Obviously, in this case, Remoting, SSH and Enter-PSSession aren’t going to be much help here.

Want to configure a server, well, then PowerShell with Remoting is probably your best tool. You could use a text based console session with SSH or Enter-PSSession, but that might be a bit too much. If you were working with say configuration files, maybe jumping to a full text session might be worthwhile.

The final situation is a complex but common one. You have a single server, maybe its hosted by a provider like Azure; your client doesn’t have any reason to trust that server, there isn’t a shared PKI to support server authentication. SSH in this situation will shine, chances are it will be easier and more secure. Either you are going to switch down to HTTP, trust a self-signed certificate or tell PowerShell to ignore CA validation errors. This is a great time for SSH to shine.

Side Note: It would be nice to see a PowerShell CMDLet to help verify Remoting certificates and even configure trust.

In each of these situations, a different remote management protocol was required. Where as in the past, we had to make do with three options (PowerShell with Remoting, Enter-PSSession, RDP) we have now gained a fourth (SSH).

A Whole New World…

I want to point to a post by Mattias Geniar, How SSH In Windows Server Completely Changes The Game. Mattias has put together an exceptional post on why SSH is a game changer in his mind.  

I feel that the introduction of SSH opens up more administration possibilities, there is an SSH client for Android, from that I could be managing my servers in minutes something that I can’t easily do that right now (well there is the PowerShell web page thing). 

The introduction of SSH also opens up opportunities for Windows in environments that are currently predominately OSX/Linux/Unix, which has to help in the adoption of products like Nano (but alas Nano will miss out this time round) and the IOT versions of Windows.

Remoting isn’t going to go away just because SSH support has been introduced. I wasn’t rejoicing the death of Remoting when SSH support was announced, I was rejoicing that progress is being made in cleaning up the whole landscape, starting with compatibility and trust. I have always seen Remoting as a successful failure, we have proved the idea works, now it is time to start working on something that is better. I really hope to see a more improved remote management protocol coming out from Microsoft over the coming years, something that is actually cross platform and easy to integrate. Until then, we have three exceptional management protocols in tool kit to use.

Kieran Jacobsen