“For an "R2," Server 2008 R2 delivered a fairly impressive list of new goodies for Active Directory (AD) techies.  One of the most important of those goodies was Active Directory’s new support of PowerShell scripting, in combination with 73 new AD-related PowerShell cmdlets and their GUI cousin, a brand-new administrative tool for AD called the Active Directory Administrative Center (ADAC).”

“I really like some of the new PowerShell cmdlets and really like the ADAC GUI tool, so I was pleasantly surprised to learn that you can fairly simply bring R2’s PowerShell cmdlets and ADAC to any Active Directory running pre-R2 domain controllers.  In fact, you can even retrofit Powershell and ADAC support on domain controllers as far back Windows Server 2003.”

Check the details in the latest newsletter from Mark Minasi.

Ken Tamaru is working on a new verson of RichCopy. It will be released in a couple of weeks.

We have been redesigning the File Server Consolidator website over the last week.

With the new homepage we want to give visitors a better impression of what File Server Consolidator is and its benefits.

The new design is a lot more open and friendly, has real screenshots of the product, and has much less text.

What do you think? Leave your comments below.

Photo Credits:jurek d.

One of the most often used roles of Windows Server is the File Services role. It provides a secure and easy way for users to share files and data. Without precise rules from the beginning, shared folders can become chaotic and finding data can become difficult and time consuming. Manual folder makeovers can be done in very small environments, but can quickly become impossible as a company grows. File Server Consolidator has been designed especially for those situations. It makes it possible to improve the folder structure when manual adjustments are not efficient. This article describes the steps for file server migration using File Server Consolidator.

Install the File Server Consolidator on a server

File Server Consolidator (FSC) is a multi-layered solution. It consists of a database, an FSC server and FSC clients. First, you have to decide on a server that will host the server part. FSC server runs as a Windows service and saves the data to a database. For user client deployment the Click Once technology is used. There are two possibilities to deploy the user client application.
Once you have a server ready that will host the File Server Consolidator service, and have decided on the client application deployment, then you can install the FSC. The installation is fully automated with the installation program and can be done in a minute.

Create and configure a new project

When you first run the administration client it will create a new FSC project for you. You can also create additional projects as needed. Each project should have a clear, descriptive name.
A very important decision that has to be made is how you are going to organize security on the new folder structure. File Server Consolidator enables very flexible folder security settings.
You can redesign as many shared folders as you wish. Before users can work with them in FSC you have to define them in the project as source locations.
Although you can leave it up to users to create the new folder structures as they like, it is recommended to define the initial structure before you start the project. You have probably decided to rearrange the old shared folders because they were not optimal. You don’t want to end up with a similar messy structure.
Together with the new location of their data users can decide also who has access to that data. To be able to do that FSC needs a list of all users who should have access to new folders. Administrators can manually add users one by one or import them from a text file. The input file can be prepared in Excel or any text editor.

Users define where the data should be migrated

This is organizationally the most difficult step. It is also the most time consuming. The ball is now in the end users’ hands. They have to start the user client application by clicking on the link that was sent to them by administrators.
The user client is very intuitive and easy to use. On the left side is the existing folder structure and on the right side is the new folder structure. Users move existing shared folders to the new structure and select their colleagues that should have read access and those that should also have a possibility to make changes.

Create and verify migration scripts

After the users have indicated where in the new folder structure they want to have their data you can create migration scripts. File Server Consolidator uses script templates to create the migration scripts. Make sure to review the scripts and also test them!

Backup data

It might happen that after the migration a user notices that he forgot to mark some of his data for migration, or he might change his mind and want to copy the data to a different location.
Before you run the migration scripts and migrate the data make sure that you have a valid backup. In case something goes wrong you might need it to recover to the previous state. This step is even more important if you are not copying, but moving the data to the new location.
It is not enough to only make the backup. It must also be tested!

Run the migration scripts

Finally, you can run the scripts and migrate the data. When you run the scripts make sure you run them in the right order.

Photo Credits:Jim Linwood

You can’t give some user groups permissions on a folder that hasn’t been created yet, or when the group doesn’t exist.

Scripts for data restructuring that File Server Consolidator generates have to be run in the following order:

1) Run the script that creates user groups in the Active Directory;
2) Execute the script that creates folders on the new file server;
3) Run the script that grants permissions on folders created in second step to user groups from the first step;
4) Run the script that will copy the data from old folders to the new ones.

Photo Credits:Marcin Wichary

When it comes to file copying you can choose from a wide variety of utilities. Rather than create another one we’ve decided that File Server Consolidator should use what already exists and has been proven to work. File Server Consolidator can work with most of the proven industry standard tools. It can be used with any tool that can be run from the command line and can accept appropriate parameters. Out of the box it generates scripts that make use of RoboCopy, ICACLS and PowerShell to migrate data.
Additionally, you can prepare templates that use some other utility to prepare scripts for Active Directory security group creation, creation of folders, and assignment of permissions and file copying.
To prepare a template for your favorite tool write a command that would execute the tool and perform some operation. Then replace parts of the command with place holders described below. During the script creation File Server Consolidator will pick the template and replace the placeholders with real values.
Example of a template:
icacls "[[TargetLocationPath]]" /grant "[[DomainName]]\[[SafeTargetLocationPath]]_R":(OI)(CI)(RX)
Placeholders that can be used with File Server Consolidator:
[[TargetLocationPath]]

This placeholder is used to mark the position where FSC should put the path to a folder in new folder structure. It is used in templates for creating new folders, copying data to new folders, etc.

[[SourceLocationPath]]

When File Server Consolidator creates scripts it replaces this placeholder with a path to a folder on an existing file share. It is used in templates for copying source data to new locations.

[[TargetLocationOwner]]

This place holder is used to mark where in the script FSC should put new folder owner’s user name.

[[ExcludedLocations]]

This term tells FSC where to put paths of source locations that must be excluded from the migration process.

[[ExcludedSublocation]]

This place holder will be replaced by one excluded sub location. It is used for creating a group of excluded source folders.

[[ParentContainer]]

With this place holder you can indicate where in the script FSC should put Active Directory container. It is used in templates for the creation of security groups in Active Directory.

[[DomainName]]

This term represents the target domain name.

[[SafeTargetLocationPath]]

Not all characters can be used to name Active Directory security groups. If you need a name of a target folder with the forbidden characters removed you can use this placeholder. FSC will take the target folder name and replace all forbidden characters with underscores. It is used in script templates for security group creation and in script templates for assigning permissions on target folders.

[[UserName]]

This placeholder is used in scripts that add individual users to Active Directory security groups. When creating a script FSC replaces this term with user’s user name.

Custom defined templates

In addition to the script templates provided with File Server Consolidator you can also use your own templates.
To replace a template that is included in FSC start a text editor and open the App.config file in the administration client. In this file you can find one line for each of the migration scripts. Find the template you would like to change and replace it with your version.
Whether you use the templates that are included with File Server Consolidator or your own, you have to test the resulting scripts in your own environment!

There are two FSC Client applications:

· User client: It allows end users to show where in the new folder structure they want to migrate their data. It is very simple to use. It gets deployed with ClickOnce technology. The Administrator sends users a link for starting the application.

· Administration client: This application is intended to be used by administrators. With this client administrators can create new projects, adjust project parameters and create migration scripts at the end of the project.

File Server Consolidator uses ClickOnce technology for user client deployment. This technology makes it possible to run a project without installing any additional software on client computers. Administrators can send users a link via e-mail, which they use to run the application.

Client deployment is possible from a shared folder or from a web server. Deployment from a shared folder is simpler, because there is no web server configuration necessary. One drawback is that ClickOnce does not allow adding parameters to a link for launching the client application. The result is that we can concurrently collect the data only for one project; however, this project can still contain multiple source servers and shared folders. This approach is more appropriate for various work group scenarios.

For larger projects it is better to use a web server to deploy user clients. The configuration is a little more complex, but allows us more options. With this approach the link for starting user client can contain parameters with project identification number and FSC server name. It makes it possible to concurrently reform many shared folders using the same FSC server. Like with the first approach we can also rebuild the structure of more than one file server.

Whether you use file shares or web servers to deploy the user client, all that a user needs to do is click on a link sent from a project leader.

Photo Credits:henning

File Server Consolidator is very flexible when it comes to folder security organization. Each folder or sub folder can have different security settings. But usually the simplest security systems are the best. The default setting in FSC is that security settings are defined only on the first level of user defined folders. All sub folders inherit security settings from the parent. At first this approach might look restrictive, but it often proves itself useful. For example, there could be problems when a child folder has different security settings than the parent folder. By changing the security settings on a parent folder an administrator can also reset the settings on child folders. That means that a child folder could have completely different permission settings than it should have. It can quickly happen that a user doesn’t have access to data that he should have. It can be even worse if somebody gets access to the data that he shouldn’t have! Important information can quickly leak.

That is the reason the FSC by default allows security settings only on one level.

Windows with NTFS permissions allows very detailed permission definitions, which can be granted to users or user groups. This flexibility is useful, but we rarely need that many options. Too many options can quickly become an obstacle. An inexperienced administrator, or a user, will be overwhelmed with that many options. The FSC enables folder owners to decide who has certain permissions to data by themselves. For this to be possible permission granting in FSC has to be as simple as possible. The folder owner has exactly two options: add a colleague to a group with read access, or add a colleague to a group with write access. If a person is in none of those groups he doesn’t have access to the data in that folder. Users in the write user group can create new folders and files, change them and also delete them. Users in the read only group can only read data in that folder. They cannot make any changes.

For each folder with defined security settings FSC creates two security groups, then puts users as members of those two groups. When permissions get granted on folders they are not granted to individual users, but rather to those two groups. This approach brings several benefits. First, security settings are much clearer. Namely, there are no mixed security settings for groups and different users. Another big advantage is that for granting or revoking permissions from users, the administrator does not need access to the file server. It is enough that the administrator has access to the Active Directory Users and Computers Console, where he can add or remove a user from the relevant group. This makes user provisioning in larger systems much easier.

Photo Credits:Syntopia

Initial structure of the new folder organization is very important. The reason you are thinking about rearranging your shared folders is likely because they are chaotic, the data is mixed up or saved to wrong places. You probably do not want to have the same problem in the new shared folder structure. There is no single right base structure. Construct the base folder structure in such way that your users feel comfortable using it, keeping in mind that not every user will be satisfied with the structure you select. The more users take part in the decision process the harder it is. Get together a small group of key users and decide on folders that will form the base for the new structure. Keep in mind that the new structure should make it easier to find the data, so be careful to not make the structure too complicated.

Pay the same amount of attention to rules for naming folders. When users start creating their folders they need simple and easy-to-use rules in place for folder naming. This is especially important in multilingual organizations. Sometimes it is very difficult to understand somebody that speaks your language, but with a foreign accent. Imagine that you are an administrator and a user asks you to change permissions on a folder. It is very important that you understand correctly what he wants. It is even more difficult when a user asks you to change permissions on a folder that is in foreign language. The folder name might even use characters you don’t have on your keyboard! In global organizations it might be easier if you have defined one language as the official corporate language. In that case, you can require that all folders that should have defined permissions must be written in that language. For folders deeper in the structure that inherit security settings from their parents you can allow users more freedom.

When File Server Consolidator prepares scripts for folder rearrangement it also creates scripts for building Active Directory security groups. It builds security group names from actual folder names. As you can use characters in folder names that are not allowed in Active Directory it has to replace those characters with underscores. The administrator’s life will be much easier if you try to have AD security group names and folder names as close as possible. Keep that in mind when you prepare folder naming standards.

However you decide your standards to be, keep them as simple as possible.

One of the promised features of RichCopy is its ability to schedule file copy jobs. It would be nice if you could synchronize your folders or migrate a file server overnight when nobody is using the files. Unfortunately, it looks like the version 4.0.217 of RichCopy has a bug that prevents it from delivering the promised functionality. When I tried to create a schedule for one of my jobs the job was run immediately or not at all. Even if the schedule functionality in RichCopy worked, it would only allow you to run the task once at specified time.

RichCopy and Windows Scheduled Tasks

Fortunately, it is very easy to use RichCopy from the command line. If you combine command line execution with Windows Scheduled Tasks you have a winning combination and you will have plenty of options to run your RichCopy jobs. You can schedule it to run daily, weekly, monthly, only on specific days in a week, when you log-on, etc.
To schedule a RichCopy job go to Start/Control Panel/Scheduled Tasks and start the Add Scheduled Task wizard.

• When asked to select the program you want to run, scroll down to RichCopy or click the Browse button and find it on disk.
• Select the desired schedule.
• Before you finish the Wizard select the check box for "Open advanced properties for this task when I click Finish." The job properties window will appear.
• Go to edit box Run and append command line parameters at the end of the line.

RichCopy is a free and powerful tool for copying files and folders. It has a nice GUI and can also be run from the command line. The command line option is very useful for scripts and batch operations.

All of the power and good stuff about RichCopy breeds a whole lot of complexity. The command line option has many possible parameters. The parameters are listed in the help file, but only with a short description, which makes it hard to manually construct the copy command.

GUI to the rescue

RichCopy’s GUI has a very nice, but hidden feature for building command line parameters. Following the steps below you can set all the options in the GUI and then generate the command line parameters.

1. Switch to Advanced View
   Open menu option View and select Advanced.
   
2. Select source and destination folders
3. Select appropriate options in the File copy options dialog
4. Select Others in the tree view
   
5. Copy command line parameters
   

Robocopy or “Robust File Copy” is a powerful tool used by many Windows system administrators to copy files and folders. It was originally available as part of the Windows Resource Kit and is now a standard feature of Windows Vista, Windows 7 and Windows Server 2008.

The powerful tool finally has an even better alternative – RichCopy. This is a free tool made by Ken Tamaru of Microsoft. RichCopy has been used internally at Microsoft for years and now they have released it to the public as a free alternative to Robocopy.

Performance improvements

The best feature of RichCopy is its fast performance, thanks to its ability to use multiple threads. It can be significantly faster than XCopy and even Robocopy. Rather than copying the files serially, RichCopy can copy several files in parallel. A user can specify how many threads to use for directory search, how many files to copy in parallel and how many threads to use for each file. With the right settings the time needed for a file copy operation becomes minimal.

Support for low bandwidth and high latency networks

RichCopy was designed to support low bandwidth and high latency networks through its built-in powerful error handling mechanisms. A user can specify how many times the program should retry to copy a file and how long should it wait between tries. Users can also select which errors should immediately cancel the copy process. For example, it might make sense to retry to copy a file from a temporary unavailable location, but there is no point to retry if access to the file has been denied.

Built-in timer

The best time to copy files is usually when users don’t need them. With most other tools that means you have to stay late and copy the files when everybody else has left the company.
RichCopy has a built in timer that lets you define the copy operation during the day and then schedule it to run overnight automatically.

Pausing, resuming and retrying a copy operation

The copy operation can be paused and retried in RichCopy. This feature comes in handy when copying files over unreliable network connections. In case of problems the process can be paused and then resumed when the problem has been fixed.

Various copy methods

RichCopy allows for a variety of different methods to copy files. With the Purge method you can mirror source to destination folders. When files or folders get deleted on the source, RichCopy will also remove them from the destination.
When the Move method is used the files at the source location will be deleted after they are successfully copied to the destination.
Another available method is to consolidate different sources.

Filtering

RichCopy offers powerful mechanisms to filter files and folders that should be included or excluded from the copy operation. Users can specify various conditions for file names, attributes and creation dates, and can set conditions to copy only files changed or accessed within a certain timeframe.

Graphical user interface

Unlike Robocopy, RichCopy has a nice GUI. The organized GUI makes configuring the copy operation a simple task. All the options available in GUI can also be used from command line. The command line parameters are listed in the help file.

Log settings

Operations can be logged to a text file and analyzed afterwards. There are many options that let users define how detailed information in the log should be.

RichCopy can be downloaded from the Microsoft Download Center.

Photo Credits:Gaetan Lee

Brian has just finished a monthly report. Before he sends it to his boss, he asks a coworker, James, to review it:

B: “James, would you please take a look at my monthly report?”

J: “Sure. Where is it?”

B: “It’s in the Drafts folder on the G drive.”

J: “Are you sure it is on drive G? I don’t see the folder named Drafts. There are only budget related folders.”

The company Brian and James work for uses many file servers. Each of them publishes many pools with folders and files. To access the folders and files more easily, users map them with different drive letters.

Brian and James both used letter G to map a network drive, but they used it to map two different network drives. No wonder they each see a different picture when they open their own drive G!

Have you ever been in the situation like that? Some users who work on many projects make use of almost all letters in the alphabet! Can you imagine how difficult it is to collaborate in an environment like that?

A time comes when you have to start with folder reorganization on your file servers. This is also the right time to standardize how network drives get mapped. When all users use the same drive letter to access a shared drive it is much easier to refer others to a specific file.

Which Letters to use for Standard Network Drives

This largely depends on the level you managed to simplify your shared folders. In general it is better to use a letter you can associate to the purpose of the drive.

Most organizations offer users a private space on a file server, known as “home folder”. You can automatically map each user’s home folder as drive H (for home). Some organizations prefer letter P (for personal).

If you managed to consolidate all your shared folders to one folder pool, you can map it with letter G (for group) or S (for shared). Some companies use letter W (for work-group).

Organizations that utilize corporate identity formats can prepare a special folder share with templates and map it as drive T (for templates). It can contain standard templates for memos, fax, orders, presentations, etc. for company-wide use.

If you haven’t used the letter S to map a shared group drive, you can use it to map a drive with useful programs (S for software). You can also use letter U (U for utilities).

Standardize Drive Letter for CD and DVD Drives

It might be useful to standardize a drive letter for CD or DVD drives. All computers could be set up to have a CD or DVD drive available under letter X. This could simplify life for authors of installation scripts. It is also convenient for users because a DVD drive will be under the same letter on all computers, independently of the number of local hard drives and partitions.

Lets say you have decided to put your file servers in order. You are going to migrate them, consolidate them or just restructure the folders. We IT professionals like it when it is not necessary to involve users. We like to buy better, bigger, faster machines and move data there, hoping the only thing the users will notice is better speed and more space. There is nothing wrong with that. But do you really have to move all the data? Is it really necessary to keep all those files forever? There might be files that nobody has opened for years! Wouldn’t it be great if you could just delete them? But how do you know if you can delete a file that hasn’t been opened for eight years? It might make sense that you can, but what if the law requires that you have to keep it for ten years? Unfortunately, only the owner of this file knows that information, and so it becomes necessary to involve users.

Involve Users

When you really want to clean up your file servers you have to involve users. They have to give you the information on which folders should be preserved and where they should be placed. I know it sounds like a big hassle if there are many users, but you do not have to work with each user individually. Instead, meet with key users or department heads and explain the details to them. Provide them with all the information material they need to inform others, then ask them to forward the information to their subordinates.

Get Management to Support You

As with all important changes in the company, it is very important to have top management support. Explain to them the problem you are solving and the benefits of the changes. Usually it is not difficult to persuade management to give their support. They also feel the pain every day when they try to find files on file servers. Users will generally take the project much more seriously if top management introduces it to them. It is a good idea to prepare an introductory e-mail message and ask the manager to send it to all the users.

Carefully Prepare the Timeframe and Deadlines

You have to give the users enough time to prepare the information you will need. When deciding on a deadline, keep in mind that it usually takes several weeks to get all the information. Some users will be very busy on their projects. Some will be out of the office and won’t be able to prepare the information for you as quickly as you would like. On the other hand, you don’t want to give them too much time. The more time you give them the more they will procrastinate. Often people collect the information you need only after being reminded several times. You should be prepared to receive most of the information just before the deadline. Schedule the deadline at least a week before the folder restructuring will actually take place. Expect also that some users will miss the deadline. Of course, no data should be lost and you will have to migrate their folders too.

Keep Users Informed

Inform the users regularly about the status of the information collection. If possible, display graphically which folders will be migrated and which folders will be deleted. If you can prepare a tree view of the source folders it will be very illustrative to them. Mark bold all the folders for which you have received the necessary information. This will get the users’ attention when they see that a folder they use is not going to be migrated. Then they will make sure you get the necessary information.

Make a Backup and Test It

No matter how much time you give to users, there will always be some files that they won’t tell you about. Make sure to make a backup to tape or other media before you move the data. It is very important to test that backup. If you are going to delete the source folders after the folder reorganization you have to make sure you can get the data from the backup.

Imagine a company that has decided to setup a file server. After the file server is installed and a share is set up, the users create several folders, where they place their files.

With time the users notice that file sharing is easier when the files are stored on the file server, rather than stored locally on their individual computers. In addition, the IT department runs a backup every night. The number of files and folders constantly increases; therefore, it now takes a great deal of effort to find the correct folder.

Additionally, users quickly find out that it is useful to prefix their folders with numbers. This way the folder appears right on top of the list. Shortly, the folder structure looks similar to the one below:

They quickly find out that other characters are even more effective. The folder list becomes like this:

Soon the filer server content becomes a muddled set of folders, with several folders serving the same purpose. It gets harder and harder to find the folder one needs.

How to prevent this problem

The solution to muddled file server content is quite simple. Do not permit users to create first level folders by themselves. They should always ask when they need a folder on that level.

In some environments it might be even better to fix the first level folder structure. Get together all the key users and have them decide on the folders that should be on the first level. After that, new folders could be added to the first level only exceptionally. Users must decide which first level is the most suitable for their data and then create a new folder beneath. All new folders must be created on the second level or lower.

What if you already have a file server in need of folder restructure?

Eventually you will have to grasp the nettle. Remember, many users will resist any change. For that reason it is essential to have top management support.

Make it easy for users to give you information on which folders they use and how they should be restructured. For smaller file servers you can create a template in Word or Excel and ask them to fill in the information. For each folder on the new server they should send you the name of the folder, where should it be placed and which folders from the old server must be migrated to the new folder. Combine the information that users send to you in a spreadsheet and periodically send it back to them. If they notice that some of their folders are not marked for processing they should send you that information.

With some effort you can transform that spreadsheet to a script that can be used to copy the old file server to a new one.

For larger file servers you might need to consider a tool designed for folder restructuring.

Windows File Server gives you many possibilities to configure permissions. You can have one set of permissions on a parent folder and another set on sub-folder(s). You can configure some sub-folders to inherit permission from the parent and the other sub-folders to have completely different permissions. You can gather users to local groups or domain groups and then grant permissions to those groups. You can also grant permissions on folders to individual users. You can use any combination of local groups, domain groups and individual users to define permissions.
With so many options available, what is the best way to manage security on file servers?

I’ve collected some best practices I would like to share with you.

Use domain security groups to manage folder permissions.

For each folder create two domain security groups. One group for users with read only permissions and another group for users with read and write permissions. Use only those two groups to manage folder permissions. When a user needs write permissions put him or her to the group that has write permissions. If he or she needs only read permissions, then put him or her to the group with read only permissions. Users that are not members of any of those two groups do not have access to that folder. A nice benefit of this best practice is that administrators can manage permissions without touching file servers. They just add or remove users from security groups in the Active Directory.

Always use the same pattern for group names. One possible pattern is “ServerName-ShareName-FolderName-GroupType”.
An example of a group name would be “UserFiles-Common-TopQualityDeviations-RW”.

“ServerName” is the name of the server where the folder is located.
“ShareName” is the name of the share on the file server.
“FolderName” is the name of the folder, where you replace spaces with underscores or use Pascal Case.
“GroupType” describes the permissions that user in that group will have on that folder. Use letter R for read only permissions and RW (or only W) for read and write permissions.
If you decide to use only one share for all the files you can omit the share name part.

Define permissions only on one folder level.

Often users want some colleagues to have read access on a parent folder. For some sub-folders they would like to give some of those users write permissions. For some sub-folders they want only two or tree people to have access and ban everybody else.
Don’t fall into that trap. It takes only a small amount of inattention when you change permissions on a parent folder and permissions on sub-folders will be reset. Some sensitive information could be disclosed and you will be in trouble.
Define permissions only on one folder level. All the files and folders beneath that folder inherit those permissions. When users demand different permissions for a sub-folder move that folder one level up and define unique permissions for that folder and all its sub-folders.

Define the “owner” of each folder

Each folder should have an owner. The owner is a person that is responsible for the contents of the folder. This is also a person that has to approve all security changes for that folder. When some users ask for access to a folder, administrators should ask the owner to approve the change before they make it. It would be even better if you have the user ask the owner to request the change.
Put the owner information to the comments field in the Active Directory security group. This way you don’t have to maintain a separate list of folders and owners. Administrators have the information readily available when they open a group properties window.