|
|
I have a need to encrypt traffic from a legacy application hosted on an AIX
server to windows clients. The information I have read seems to point at
using certificate based IPSec to accomplish this or using ISA server as an
IPSec proxy. Has anyone implemented either of these solutions, information
has not been easy to find on how to do this.
The application runs over Telnet and the clients are 2000 or XP.
|
|
I have never tried to set IPsec up on AIX, but here are a few links that may
help:
http://inetsd01.boulder.ibm.com/pseries/en_US/aixbman/security/ipsec_planning.htm
http://www.redbooks.ibm.com/redbooks/pdfs/sg246622.pdf
Jason
"OJG" <OJG[ at ]discussions.microsoft.com> wrote in message
news:F602A271-4C66-42F9-B7B6-CD978E689A59[ at ]microsoft.com...
[Quoted Text] >I have a need to encrypt traffic from a legacy application hosted on an AIX
> server to windows clients. The information I have read seems to point at
> using certificate based IPSec to accomplish this or using ISA server as an
> IPSec proxy. Has anyone implemented either of these solutions, information
> has not been easy to find on how to do this.
> The application runs over Telnet and the clients are 2000 or XP.
|
|
Here are the directions from IBM's tech support just in case anyone else has
the pleasure of doing this. I have tested this and it works we are now
working on using certificate based encryption to subnets rather than
specifying both sides of the tunnel.
Prior to setting up IKE tunnels on AIX, you'll need to install the ipsec
filesets from your AIX base media:
bos.net.ipsec.rte
bos.net.ipsec.websm
bos.net.ipsec.keymgt
You'll also need to install the "bos.crypto-priv" fileset from the AIX 5.3
Expansion Pack CD.
Since you'll eventually use certificates, you also want to install the
"gskta.rte" package from the same expansion pack CD.
Thanks!
--
Thanks!
Tu Vo ~ AIX Technical Support ~ T/L 523-4248
For more information regarding AIX support:
http://techsupport.services.ibm.com/guides/handbook.html
=====================================================================
How to configure a sample IKE tunnel between AIX 5L and Windows 2000
=====================================================================
This document will show you how to configure a sample IKE tunnel between
AIX 5.2 and Windows 2000. The tunnel will use pre-shared key of 12345
------------------------------
I. Pre-configuration checklist
------------------------------
- Write down IP address of AIX machine
- Write down IP address of Windows 2000 machine
- If there is a firewall in between:
- Open these ports on the firewall:
- UDP port 500 (for ISAKMP traffic)
- UDP port 1011 (for the Session Key daemon)
- UDP Port 4001 (for the Session Key daemon)
- Protocol 50 (for ESP traffic)
- Protocol 51 (for AH traffic)
- Verify that you can successfully launch and access WebSM on AIX
- WebSM is needed for AIX IKE configuration
--------------------------------------------------------------
II. Install AIX IPSec software and put on latest IPSec patches
--------------------------------------------------------------
- Install the following IPSec filesets from an CD1 of your
AIX 5L Base Media:
bos.msg.en_US.net.ipsec
bos.net.ipsec.keymgt
bos.net.ipsec.rte
bos.net.ipsec.websm
- Check for latest patches to above files and install if
applicable
- AIX fixes can be downloaded from the following URL:
http://www-1.ibm.com/servers/eserver/support/pseries/fixes
--------------------------------
III. Configure IKE tunnel on AIX
--------------------------------
Note about WebSM: Once you access WebSM, you will see that there
are drop down menus at the top of the window. Below the drop down menu
are various icons. Below the icons are two frames. The left frame is
the Navigation Area and will be referred to as such in this document.
The right frame will be referred to as the Main Window.
- Access WebSM
- Under the Navigation Area frame on your WebSM window, click
on the plus sign next to "Network" to expand it
- Expand "Virtual Private Networks (IP Security)"
- Click on "Overview and Tasks"
- On the Main Window, click on "Start IP security", take
the defaults, click OK, click Close on the "Working..." window
- Click on Internet Key Exchange (IKE) Tunnels in Navigation area
- On the drop down menu at the top, click on Tunnels; select New;
select Basic Tunnel Connection (wizard)
- A "Configure ..." wizard pops up at Step 1; click Next
- At Step 2, enter a tunnel name; leave "Host" selected for local
and remote system since we are creating a Host-Host Tunnel;
click Next
- At Step 3, choose the correct IP address for your local ID and
enter the IP address of the PC for the remote identifier; click
Next
- At Step 4, leave "Authentication method" at Pre-shared key;
enter a preshared key of "123abc" for example; click Next
- At Step 5, change the hash algorithm to HMAC SHA, and change the
encapsulation mode to Transport; Leave the other selections at
their defaults; click Next
- At Step 6, click Finish and OK to close the wizard
- Back on the WebSM window, in the IKE Tunnels window, you will
see two entries corresponding to the Phase 1 (P1) and Phase 2 (P2)
tunnels for the IKE tunnel you just defined
- The P1 tunnel has a "-" sign to its left, indicating that it
has been expanded to reveal the P2 tunnel below
-------------------------------------------------------------
IV. Preparing Windows 2000 for IPSec IKE tunnel configuration
-------------------------------------------------------------
If this is the first time that IPSec has been configured on the Windows
2000
box, please follow instructions in the Microsoft doc below to create a
custom MMC console:
http://www.microsoft.com/windows2000/techinfo/planning/security/ipsecsteps.asp
You only need to perform the steps in the "Preparing for Testing" section,
which involves:
- Creating a custom console
- Enabling audit policy
- Configure the IP security monitor
You do NOT need to go through the "Using a Built-in IPSec Policy" section.
save your custom console by clicking on the "Console" and selecing "Save".
You'll also want to ensure that the "IPSEC Policy Agent" is enabled.
Here's how to verify the Windows 2000 IPsec Policy Agent:
- Click on Start Menu -> Settings -> Control Panel -> Administrative
Tools
-> Component Services
- This will pull up the Component Services Console
- Click on "Services (Local)" on the left hand side
- Look for "IPSEC Policy Agent" on the right hand side and make sure
that
its status is "Started" and startup type is "Automatic"
- Consult Windows 2000 documentation if you have problems with this step
---------------------------------------
V. Configure IKE tunnel on Windows 2000
---------------------------------------
- On the left pane of your custom console, navigate to "Local Computer
Policy"
and expand it
- Navigate to "Computer Configuration" and expand it
- Navigate to "Windows Settings" and expand it
- Navigate to "Security Settings" and expand it
- Click on "IP Security Policies on Local Machine"
- Click on the "Action" menu at the top of the console, select "Create
IP
Security Policy
- This will bring you into the IP Security Policy Wizard
- Click Next
- Enter a name and optional description for your security policy, and
click Next
- Uncheck the "Activate the default response rule" box, and click Next
- Leave the Edit Properties box checked and click Finish
- Make sure "Use Add Wizard" is checked, and click Add
- This will put you into the Security Rule Wizard
- Click Next
- Make sure "This rule does not specify a tunnel" is checked and click
Next
- Make sure "All network connections" is checked and click Next
- Select the preshared key choice, enter a key
- The preshared key should match exactly what you entered in your
AIX configuration; in this example, I used "123abc"
- Click Next
- Cick Add
- Enter a name and optional description for this filter list, make sure
that
"Use Add Wizard" is selected, and click Add
- This will put you into the IP Filter Wizard
- Click Next to take the default setting for source IP address
- Select "A specific IP address" for the destination IP address, and
enter the IP
address of the AIX machine
- Click Next
- Click Finish
- Now you will be back on the IP Filter List window
- Click Close
- Now you will be back in the Security Rule Wizard
- The filter list you just created should be one of the items in the
list
- Select it and click Next
- Select Require Security and click Next
- Click Finish
- Now there should be two IP Security Rules listed, the default rule
and the
one you just created
- Make sure the one you just created is checked, and the default rule is
unchecked,
and then click Close
- The database is now set up properly; you should be back on the Local
Security
Settings window
- Go to the security policy you created, and right-click on it
- This will bring up a menu,
- From this menu, select Assign
- The "Policy Assigned" column should change from No to Yes.
--------------------------------------------------------
VI. Send traffic through the newly configured IKE tunnel
--------------------------------------------------------
Note that by following the steps above, you will not be able to negotiate
from the
AIX side. This is because when Microsoft initiates, the default key life
durations
in Microsoft fall within the ranges the default policies in AIX. But if
AIX were
to initiate, Microsoft would require an exact match of the life duration
parameters
to its default values, and this will fail. It can be changed on either
side, of
course, so that the values do match, but the steps above are meant to show
the
simplest way to get a working negotiation. Also, typically AIX will be the
machine
acting as a server, and Windows2000 as the client, so this setup will work
in most
real-life scenarios.
To start a negotiation from the Microsoft side, run some sort of traffic
to the AIX host.
For instance, try to ping the AIX box from the windows DOS prompt.
Your first ping attempts will fail because the tunnel has not been
negotiated yet and
is not currently active. Your pings may fail or you may see 4 messages
about
"Negotiating IP Security."
After the initial ping failures, try the ping again, it should work now.
-------------------------------------------
VII. A note about Windows 2000 IKE database
-------------------------------------------
The Microsoft database is set up somewhat differently from our own. It
does not make a
clear distinction between phase 1 and phase 2 information, and it makes
the filter rule
information part of the tunnel information, instead of a layer below it.
Further blurring the line between phase 1 and phase 2, Microsoft expects
phase 2 to
follow immediately after a successful phase 1 negotiation has taken place.
If the
corresponding phase 2 negotiation does not happen within 60 seconds after
the phase 1
finishes, Microsoft will kill the phase 1 tunnel, and send a delete
notification to
the other side for that SA. Essentially, Microsoft is viewing phase 1 and
phase 2 as
part of a single tunnel negotiation, which it thinks has stopped in the
middle if only
phase 1 completes.
Here is a rough diagram of how to find information in the Microsoft setup.
IPSEC policy
-------------------------------------------------
| |
| |
| Phase 1 transform list |
| Phase 1 life duration |
| Phase 1 PFS |
| |
| |
| IPSec Rules List |
| | |
| | |
--------|----------------------------------------
|
| one-to-many
|
v
-------------------------------------------------
| |
| |
| Phase 1 authentication method |
| Encapsulation mode (tunnel/transport) |
| called "Tunnel Setting" |
| |
| |
| IP Filter List Filter action |
| | | |
| | | |
| | | |
-------------------------------------------------
| |
| one-to-one | one-to-one
| |
v v
------------------------- -----------------------------
| | | |
| | | Permit/Deny/Negotiate |
| Phase 2 endpoints | | Phase 2 transform list |
| Phase 2 protocols/ | | |
| ports | -----------------------------
| |
| |
-------------------------
The IP Filter List contains all the information that is needed to place a
filter
rule: the endpoints, netmasks, protocol, and port. It also contains a flag
called
"Mirror," which tells whether the filter rule should apply to packets with
the
source and destination information reversed.
The corresponding filter action is where Permit or Deny is specified. A
third
action, Negotiate, can also be specified, which is the same as an AIX
Permit rule
that specifies an IKE tunnel. The phase 2 transform list is only active
for the
case where the action is set to Negotiate.
============================
Last updated on: Feb 3, 2004
============================
Differences on windows XP:
- IPSEC Policy Agent is now called IPSEC Services
- There is no "ipsecmon" command
"Jason Popp [MS]" wrote:
[Quoted Text] > I have never tried to set IPsec up on AIX, but here are a few links that may > help: > > http://inetsd01.boulder.ibm.com/pseries/en_US/aixbman/security/ipsec_planning.htm> > http://www.redbooks.ibm.com/redbooks/pdfs/sg246622.pdf> > Jason > > > "OJG" <OJG[ at ]discussions.microsoft.com> wrote in message > news:F602A271-4C66-42F9-B7B6-CD978E689A59[ at ]microsoft.com... > >I have a need to encrypt traffic from a legacy application hosted on an AIX > > server to windows clients. The information I have read seems to point at > > using certificate based IPSec to accomplish this or using ISA server as an > > IPSec proxy. Has anyone implemented either of these solutions, information > > has not been easy to find on how to do this. > > The application runs over Telnet and the clients are 2000 or XP. > > |
|
|