Saturday, May 13, 2017

Cookbook for turning large network captures into firewall change requests

Cookbook for turning large network captures into firewall change requests

I started with 15 million captured packets. The instructions below illustrate how I reduced that data to a unique list of outbound connections. I’m sharing this in case any of you need to do this in the future.

1.    12 hour capture on source machine, writing to 50 MB capture files. Implement this via a scheduled task / script. (This is the source of the 15 million packets.)
2.    Use Log Parser on a 2003/XP machine to export capture files to .CSV files
3.    Combine all .CSV files into one file. Notepad cannot open a file this large.
type *.csv>>result.txt
4.    Use a custom written script to break the large file into separate files 1048576 lines in length. 1048576 is the largest row set Excel can use in a single worksheet. (In the second pass, I wrote a C# program to do this.)
5.    Open each file in Excel. I suggest using unique instances of Excel for each file, which cuts down significantly on delays. It helps if you have a laptop with a lot of RAM and 64-bit Office when doing this. I used this command to launch multiple instances of Excel:
for /f %x in (‘dir *.csv /b’) do start /separate “Excel” excel.exe %x

In each Excel file:
6.    Filter out all inbound connections
7.    In a new column, concatenate target IP and target port to DstIP:DstPort
8.    Use filtering to filter DstIP:DstPort down to only unique entries
9.    Combine all uniques in a single summary workbook and use filtering to reduce that to a list of uniques

At this point, I had a complete list of unique outbound connections. I needed to put this in an “ISM7 friendly” format.

Resolve host names:
10.    Use filtering to generate a list of unique IP addresses
11.    Use a custom written script to resolve each IP address to a DNS name. The script outputs a file in the format “IP,FQDN”
12.    Import the result to a new worksheet (comma delimited text file import), then use vlookup to associate host names with IP addresses in the main worksheet

Consolidate the data:
13.    Sort by target IP
14.    Manually search through the list to identify multiple connections to a single IP
15.    Identify any servers that appear to require the use of high or dynamic ports (many connections to the same IP using unique high port numbers). Most of these cases require opening 1025-65536
16.    Identify servers that require multiple specific ports and consolidate the port requirement to a single line, deleting duplicates
17.    Identify connections to AD DCs and delete them (should be covered by a “tier zero” ISM7, which is a documented standard)

At this point, I had a nearly complete list of unique outbound connections with no duplicates. All that was left was to massage this data into the firewall change request format, add the source server information, and submit the request.

- - - - -

This was a bit of work (a few hours), but not impossible. In the future, I would create an application that would do most of the heavy lifting. The application would:
•    Read in a capture file in the capture tool’s native format. There are open source C# projects that read in a variety of capture formats, including MS Network Monitor
•    Reduce the result to a list of unique targets (DstIP:DstPort) and perform this comparison in memory. This would speed up the data analysis process dramatically.
•    Once the unique target data is available, there would still be some manual human analysis that would be necessary, but the above would speed up the data processing dramatically.

If you made it this far, thanks for reading.

Smart Projector UF65 Teardown and Reassemble - Replacing the DLP chip and color wheel in a Smart Projector UF65

Topics covered:
  • Replacing the DLP chip in a UF65
  • Replacing the fans in a UF65
  • Replacing the color wheel in a UF65

Items needed:
  • Long phillips screwdriver
  • Remote control, so you can bench test the unit without attaching it to the Smart board wall mount. If you do not have the remote, you can program a Logitec Harmony remote for the UF65 (this is what I did - it worked perfectly)
  • Canned air. I usually keep a 3 or 6 pack handy, since they run out of pressure quickly and I don't like to wait.
  • Long tool with magnetic tip

  • All screws are metal into metal. These units are well made, and are meant to be taken apart.
  • These units are older and some parts are made of plastic. The plastic gets brittle. Be gentle tightening screws and leaning on plastic parts
  • Be careful with the color wheel. It is made of glass, and if you bang it or drop it, it will break. I broke one the first time I took it apart and had to wait a week for parts
  • Obviously be careful with the lamp
  • Most photos are with the projector pointing at the camera to help with orientation

Projector disassembly

1. Remove the bulb cover, the bulb, and the 4 screws in the bulb mounting chamber

2. Flip the unit over. Remove the 3 screws from the deep holes on the bottom of the unit. (You will need an extra-long screwdriver for one of them)

3. On the inputs side (power, VGA1, VGA2, etc), remove all screws

4. On the front of the unit, remove the ring from around the lens with your bare fingers. It should pop right off without breaking anything, but it's plastic, so be careful.

5. With the unit on it's feet (status lights up), gently remove the top cover. There are two wires connected to it. One is for the LED status lights, the other is for the IR receiver.

6. Gently disconnect the two connectors with your fingers. Pull by the connector, not by the wires.

7. Good time to blow the dust out of the unit.

8. Remove the EMI shield by removing each of the screws around the shield. Some screws are different lengths, so keep track of them by drawing a picture of the shield and placing the screws on the picture.

9. Remove the 8 screws holding the main board in place. Disconnect the power connector from the main board. Unstick the connector for the color wheel. Center the focusing ring so it is not obstructing removal of the main board. Again, keep track of your screws.

10. Lift the left side of the board to disconnect the DLP chip, then lift the board up a few inches. The main board is attached to the DLP chip via a slide-in style connector on the bottom of the main board, so there will be some light resistance when lifting the board up. Note that the I/O ports (VGA1, VGA2, etc) are connected to the main board, and all move as one large piece.

11. Underneath the board on the right, disconnect the two prong power connector, and unscrew the yellow+green grounding screw.

You should now be able to lift the main board up and fold it over to the left without disturbing the connectors to the left.

Replacing the DLP chip

Note: If you are replacing fans or a color wheel, you do not need to perform these steps. Skip ahead to the next section.

12. Remove the lens body holding the DLP chip, color wheel, lens, and DLP chip heat sink. Remove the screws (4). Detach the sticky air directors at the top / far side of the unit. Lift the lens body out of the unit. A reminder to be careful with this unit. If you drop it or bang it, you will likely break the color wheel.

13. Remove the heat sink.

  • The two screws holding on the heat sink are also holding springs, so when you remove the screws, be prepared for parts to fall to the work surface or to the floor.
  • The DLP chip and the DLP chip mount may also fall out, so be prepared to catch them.
Lift one side of the copper tape and peel it back.

Remove the two screws attaching the heat sink to the lens assembly. When these two screws are removed, the heat sink will be removed (nothing else holding it on), so be careful.

14. Remove the PC board holding the DLP chip, chip mount, and rubber spacer.

15. To remove the DLP chip from the chip mount, there is a lock / unlock screw. Turn the screw to the unlock position and lift out the old DLP chip.

If you are replacing the DLP chip to resolve an issue with dead or white pixels, the dead pixels should look like tiny dust specs on the DLP chip you just removed. (The new chip should appear absolutely clear).

16. Install the new DLP chip. Insert the chip into the PC board connector. Put the PC board on a firm, nonmetal surface. Gently press the DLP chip onto the PC board and turn the connector to the locked position.

17. Install IC Diamond conductive paste on the back of the DLP chip

The new DLP chip you received should have a thin pad included with it. This pad is the conductive surface that transfers heat from the chip to the heat sink.

Story time:
  • At first, I used this blue pad included with the chip.
  • When I reassembled the projector, it displayed a fully white picture, with no details at all
  • I took the projector apart and discovered the pressure of the heat sink on the DLP chip had pushed the chip out of the chip mount completely, so there was no contact between the pins and the PC board
  • I reassembled the projector, this time putting less pressure on the springs + screws holding the heat sink in place
  • Initially, the picture was fine. The next day, the picture had horizontal bars.
  • I took the projector apart again, and discovered the DLP chip had again been partially pushed out of the PC board connector
  • You MUST use a conductive material between the DLP chip and the heat sink
  • Do not use the blue pad included with the new DLP chip. It is both too thick and too dense, and the DLP chip will eventually unseat from the PC board connector due to pressure from the heat sink
  • Instead, use a pea-sized bead of Diamond IC thermal paste (follow IC Diamond instructions) to fill the gap between the DLP chip and the heat sink, and screw the two springs + screws on the heat sink in all the way. This assures maximum conductivity (heat is what kills DLP chips), and assures the chip will not become dislodged. 
  • If you see a white screen after re-assembly, it is likely the DLP chip has been disconnected from the PC board
  • If you see horizontal black bars after re-assembly, it is likely the DLP chip has been disconnected from the PC board
18. Place the rubber spacer on the lens assembly. There are two poles that hold it in place correctly.

19. Place the PC board with the DLP chip and conductive paste on top of the rubber spacer. The PC board also fits onto the two poles. It's a puzzle where the pieces fit only one way, so there's no way to get it wrong. (If you're using a hammer, you're doing it wrong.)

20. Get your screws and springs ready.

21. Gently seat the heat sink on the PC board + lens assembly and line up the screw holes. Start one screw at a time, but don't tighten them. Once both are started, gradually tighten each screw until they are fully seated. There is a few millimeters of space left for the springs to be springs (they will not be fully compressed). When tightening the screws gradually, visualize evenly spreading the IC Diamond thermal paste over the DLP chip and heat sink while equally tightening the two screws.

22.  Return the lens assembly to the projector unit. Return the 4 screws for securing the unit. (Be careful with these. Even though they are metal on metal, I stripped one of these.)

23. Return the heat shield / air directors to their original position.

This concludes replacement of the DLP chip.

Replacing the color wheel

  • It is critical that you have the correct replacement part for the color wheel. The color sequence must match, and the size of the color patches must match.
  • For the UF65 unit I was working on:
    • The color wheel was approximately 40 mm in diameter
    • It was keyed to the blue color (there is a black band on the wheel corresponding with the blue color, which is what the color wheel position sensor is looking for)
    • The color sequence, clockwise from the keyed color (blue), is blue, clear, turquoise, green, yellow, red
    • The blue portion was approximately 23mm of the diameter of the color wheel
  • It is necessary to remove the main board (already covered above) to replace the color wheel
  • It is *not* necessary to remove the lens assembly to replace the color wheel
  • The color wheel is connected via two connectors to the main board 


24. The color wheel assembly is held on by two screws. Remove the two screws, and remove the color wheel assembly from the unit.

25. Disconnect the color wheel position sensor, which is a two pin plug-in style connector. Remove by the connector, not by pulling the wires.

26. Disconnect the color wheel motor. Remove the flat ribbon-cable style connector from the connector by first pulling out the flat locking part of the connector, then gently removing the ribbon cable. Note that once the flat locking part is removed, the ribbon cable slides out easily. If you have to pull hard to remove the cable, you are doing it wrong. Also note the

27. Use the three screws on the back of the color wheel assembly to remove the color wheel motor and color wheel from the assembly. Note that the color wheel position sensor is part of the metal body, and is not removed or disturbed in this step, though you may want to use canned air or an air bulb to clean it. Keep the screws and do not remove the red dampeners around the screws.

28. Attach the new color wheel to the assembly. Be sure the ribbon cable is coming out of the assembly in the correct direction. Fully seat the three screws by gently tightening them, but don't overdo it.

29. Return the color wheel assembly to the projector unit (2 screws)

30. Attach the color wheel position sensor to the main board

31. Attach the color wheel motor to the main board. The ribbon cable connects via "blue side up." Gently insert the cable, then slide the locking slider into place. Give a very gentle pull to the ribbon cable to assure it is correctly seated.

This concludes the replacement of the color wheel.

Projector re-assembly
  • Re-assembly is the reverse of disassembly. Be careful not to break the color wheel. If it is fractured and you turn the unit on, it will fly apart and be out of balance.
32. Return the main board. Note you must attach the main board to the DLP chip PC board and lens assembly before the main board will seat fully in the unit.

33. Return the EMI shield. Note the sides of the unit are partially held in place by the heat shield via metal tabs and slots.

34. Return the top cover

35. Return the 3 bottom screws

36. Return the IO panel screws

37. Return the screws around the lamp, the lamp, and the lamp cover

38. Return the ring around the lens

This concludes the re-assembly of the projector. Connect the projector to power and test.