Data management for Civil 3D projects

Last updated: 2023-09-14

Info: What is data management for Civil 3D projects?

Data management for Civil 3D projects is the theory, standards, and methods for:

  1. Naming
  2. Organization/Structure/Division

  3. Connections of data

This includes all of the data (files and objects within those files) that can be used in a Civil 3D project:

  1. Existing condition data
  2. Design files (options, analysis, ultimate design, hydrology, right-of-way, etc.)
  3. Sheet files (Plan sets, report inserts, large format, etc.)
  4. Digital files for construction (Construction Data Packet)

Info: Why do we care about data management?

  1. Keep a design project "healthy" throughout the lifecycle of the design timeline. Your car needs oil changes, tire rotates, battery checks. Your design files need love too!
    1. Eliminate/minimize crashes
    2. Good user experience
      1. Performance (reasonable time to open/save files)
      2. Easy and quick to understand the content of files/objects on any given project
      3. Able to create new files and objects that are easily consumed by other users.
    3. Efficient
      1. It should be easy for an individual to work through a workflow and it should be easy for multiple users working throughout the lifetime of a project. There is a balance between the "fastest" way to complete an item and making sure it is appropriate for downstream uses and needs.
      2. Example: Plan/sheet printing at various milestones is standardized, repeatable, efficient, and effective.
      3. Example: Edits/updates can be completed with the minimum amount of work while staying prepared for downstream workflows.
    4. Maintain project data that is flexible to changes

  2. Deliver files to post-design business needs. Currently this is mostly the Construction Data Packet (CDP) for construction.

This is the language of project files and objects. It is not an all-inclusive list that works for every project. It is a base level of common need and a language for expansion so that when you need to create your own content, others will be able to understand it.

There are reasons why the standards are what they are. Here they are:

  • FDM 15-5 Attachment 3.1 CADDS AutoCAD Civil 3D Project Folder Structure Standard
    • Short folder names. Character count can matter depending on how deep the structure is.
    • Custom subfolders are allowed. Don't delete standard folders.
    • All current, active sheet files should reside in ProjectID\sheets. This will make creating a DSD file for plotting easier.
    • Contractor Data folders have a CD-prefix. This prevents mistakes in copy/paste functions.
  • FDM 15-5 Attachment 3.3 CADDS AutoCAD Civil 3D File and Layout Naming Standard
    • There are common design files and standards for sheet files. Review them BEFORE starting design. Your future self will thank your past self.
    • Layout names should only be 01, 02, 03, etc. This serves two purposes:
      • Makes more layouts directly accessible from the bottom edge
      • Combined with placing all sheet DWG files in the same folder, makes plotting from a DSD file through batch publish as fast as possible.
  • FDM 15-5 Attachment 3.4 CADDS AutoCAD Civil 3D Object Naming Conventions
    • Roadway alignment names need to be as short as possible.
      • Highways - Numbers only. Drop the STH, WIS, IH, US, etc.
      • County roads prefix with Cty-, local roads prefix with Lcl-. This will group all stationed highway classes together.
    • Roadway alignments are the basis of all following alignments. It is a best practice to make edgelines that participate in corridors (ETW, EPS, EGS, etc.) alignments with intelligent names to make modeling easier and fully documented.
    • Template - Highway-Side-Feature-Location. This can be expanded for directional roadways, intersections, roundabouts, curb returns, etc. Keep things short. It makes them more accessible and usable in software dialog boxes.
  • FDM 15-5 Attachment 3.2 CADDS AutoCAD Civil 3D File and Object Naming Abbreviation Standard
    • Drop vowels unless the result is unreadable.
    • Try to keep abbreviated terms to 4 characters or less.
    • Acronyms are all caps. Abbreviations are mixed case.
  • Besides files, layouts, and Civil 3D objects there are AutoCAD objects (mtext, plines, etc).
    • All Civil 3D and AutoCAD objects within a file are on a layer.
      • Standard layers are built into the WisDOT template files.
      • Files do not need to have every layer within them.
      • Custom layers are allowed, but should be prefixed with standard layers.

Things to know about layers

  1. Freezing layers is more powerful than turning layers off when it comes to XREFs. This prevents the data from being in the working session.
  2. The default layers for WisDOT Civil 3D objects is a layer group prefix followed by the name of the object. Example: The layer for a proposed alignment with a name of 10 (for US 10) would be P_Ali_10. This allows for layer control of freeze/thaw or on/off for individual Civil 3D objects. When using the WisDOT standard templates, Civil 3D Object layer assignments are automated.
  3. Layer filters are groups of layers that let you work with focused layer subsets in the layer properties manager and layer lists.
  4. The vast majority of WisDOT plotting is done through the color of the layer. (i.e. changing the color of a layer can change how it plots). Some plotting of Civil 3D objects is controlled through styles.
  5. Layer properties are a table of values that can be changed if needed. Color can be changed for different plotting effects and freezing/thawing layers can be very common.
  6. Layer states are snapshots of the layer properties of a file and it's XREFs at any given time. These states can be saved and reused to speed up changing layer states in a file. Layers and their properties

Helpful layer tools

  1. WisDOT *** Tab > Manage panel > Layers pulldown
    1. Layers - Convert Layer Colors
    2. Layers - Layer Order
    3. Layer States Palette

    4. Layers - Assign Object Layer Defaults

  2. WisDOT *** Tab > Manage panel > Xref tools pulldown 
    1. Xref tools - Load Xref on Layer
    2. Xref tools - Assign Xref to Layer

Info: When/why do we break up files/objects?

  1. Logical break out
  2. Allow multiple users to edit data concurrently
  3. Keep file sizes manageable for hardware/infrastructure
  4. Setting up data for downstream deliverables (CDP)

Info: When/why do we connect files/objects?

There are MANY reasons that are seemingly obvious. For example, to design cross-sections, you need many inputs (alignment/profile, existing ground, proposed horizontal geometry). The important things to understand is:

  1. Once you divide data into multiple files, you need to connect data together to complete certain tasks.
  2. You need access and resources to not only open the file you're working in, but all the files you're connected to at any given time.
    1. Example: If you have a 5KB alignment file that has a data shortcut to a 10GB existing surface, you are working with 10GB, not just the 5KB.
    2. Example: If you have 10x 100MB topo files, if you XREF one of them, you're working with 100MB. If you XREF ALL of them, you're working with 1GB!

Methods to connect to files in Civil 3D

  1. XREF
    1. Visual representation of entire file. Can see all AutoCAD and Civil 3D objects
    2. Basic display. You cannot access the attributes of Civil 3D objects (superelevation, surface data, changing styles)
    3. Can turn on/off, freeze/thaw data on XREF layers.
  2. DREF (data shortcut)
    1. Intelligent and dynamic connection to specific Civil 3D object types
    2. Surfaces, alignments, profiles, pipe networks, corridors, sample line groups, view frame groups
    3. Can access attributes and have different styles than the source file. Example: an existing surface can have a border only style in the source file and you can change it to a contour interval in a host file.
  3. SREF (survey data reference)
    1. These are file connections that are not XREFs or DREFs. They are similar to DREFs in that you can access attributes of the source objects, but the connections are not made through the Data Shortcuts interface.
    2. Any DWG can have connections to
      1. Survey databases/queries to see points and figures. Examples:
        1. Topo-Ex.dwg
        2. Cross section files referencing utility figures
        3. Hydro design files connecting to culverts
    3. Surfaces can have connections to:
      1. Survey databases/queries
      2. PNT/FLT files
      3. LandXML files
      4. DEM files
      5. TIN files
    4. Data Connect (Map 3D)
  4. Source DWG
    1. The file where the object has been created
  5. Host DWG
    1. A file where the object or file is consumed

Things to know about XREFs

  1. With few exceptions, XREFs should be
    1. Relative (stays intact even if project is moved or copied)
    2. Overlay, Not Attachment (prevents XREFs from getting passed on through multiple layers of references. This may be convenient in some cases, but it is not a best practice for data management since it provides more opportunities for circular references and potentially unintended load increase)
    3. Scale = 1, Rotation = 0, Insertion = 0,0,0
  2. The best way to do XREFs is WisDOT *** Tab > Manage panel > Xref Tools pulldown > Load Xref on Layer It does all the settings above AND places the XREF on it's own layer that is the name of the file.
  3. XREFs should only be saved to a file if they are often being used. To minimize the impact of large XREFs on file open, the XREF definition can be retained, but unload the XREF.

Things to know about DREFs

  1. Promote creates a copy of an object in a host file that does not stay live to the original object. This command should not be used unless this functionality is needed.
  2. The style of a DREF can be changed in a host file.
  3. If you change the name of an object in a source file, Civil 3D will keep the connection live, but it will not automatically change the name in the available Data Shortcuts or the name in host files.
    1. Toolspace > Prospector > Validate Data Shortcuts will apply new names (and check for missing objects)
    2. Inside each host file, Manage tab > Data Shortcuts panel > Validate Data Shortcuts will apply new names (and check for missing objects) in the host file.
    3. If you can avoid renaming objects after they are being consumed in other files, that can save you some headaches.
  4. If you need to redefine object geometry, it is not necessary to delete the object and create a new one.

Detailed documentation

  1. Project data workflow map
  2. Edgeline data management

  1. There is no one way to do data management that fits every single project. We have too many different scope possibilities for design projects for that to work.
  2. The biggest part of delivering a design and plan is the decision-making process of developing the design, not the drafting of the sheets.
  3. You don't look up standards after you design a project. You look up standards BEFORE you design. The same concept should be brought to data management (structure, naming).
  4. Develop a plan (storyboard) for your work before beginning your work.
    1. What existing condition data do we need?
    2. What design do we need?
      1. Corridor(s)
      2. Drainage
    3. What sheets will we need?
    4. What CDP files will we need to deliver?
  5. The first time you do something in Civil 3D, you should NOT be thinking "what shortcuts can I take?". The first time you should be following steps exactly. Only after going through an entire design process, should you be looking for shortcuts.
  6. Files that are consumed by many different downstream files need to have as little connected files as possible to avoid circular references and longer load times.
    1. The following files should NEVER have any permanent XREFs in it. They get consumed by so many downstream files.
      1. AliProf-*.dwg
      2. Srfc-Ex.dwg
      3. Topo-Ex.dwg
  7. Data needs to flow in one direction to avoid circular references.
  8. XREFs and DREFs only cause circular reference problems if they are saved into the host file database (i.e. the file is then looked for when host file is opened.). You can reference a file and unload or remove the reference before you save it.
  9. Don't be afraid to:
    1. Ask for assistance.
      1. CAE Support
      2. Methods Development
      3. Experienced local users
      4. Google
    2. Push buttons
    3. Create/edit/delete data
    4. Build out the entire flow of files with preliminary data and then make updates to them as the design progresses
      1. Example: AliProf > Corridor > Refinement Surface > Cross Sections

We'll work on getting your file back up and running, but the most important thing to remember is even if you fix the file, if you keep doing the same thing (same file size, same DREF/XREF, same workflow, same button pushes, etc.), you should expect the file to break in the same way again.

"Insanity: doing the same thing over and over again and expecting different results." - Albert Einstein

Requirement: You NEED to change SOMETHING.

  • Are my files too big?
  • Is there a bad/huge XREF?
  • Maybe the steps I took aren't good for some reason?