|
LLVM
8.0.1
|
Drive the analysis of interleaved memory accesses in the loop. More...
#include "llvm/Analysis/VectorUtils.h"
Public Member Functions | |
| InterleavedAccessInfo (PredicatedScalarEvolution &PSE, Loop *L, DominatorTree *DT, LoopInfo *LI, const LoopAccessInfo *LAI) | |
| ~InterleavedAccessInfo () | |
| void | analyzeInterleaving (bool EnableMaskedInterleavedGroup) |
| Analyze the interleaved accesses and collect them in interleave groups. More... | |
| void | reset () |
| Invalidate groups, e.g., in case all blocks in loop will be predicated contrary to original assumption. More... | |
| bool | isInterleaved (Instruction *Instr) const |
Check if Instr belongs to any interleave group. More... | |
| InterleaveGroup< Instruction > * | getInterleaveGroup (const Instruction *Instr) const |
Get the interleave group that Instr belongs to. More... | |
| iterator_range< SmallPtrSetIterator< llvm::InterleaveGroup< Instruction > * > > | getInterleaveGroups () |
| bool | requiresScalarEpilogue () const |
| Returns true if an interleaved group that may access memory out-of-bounds requires a scalar epilogue iteration for correctness. More... | |
| void | invalidateGroupsRequiringScalarEpilogue () |
| Invalidate groups that require a scalar epilogue (due to gaps). More... | |
Drive the analysis of interleaved memory accesses in the loop.
Use this class to analyze interleaved accesses only when we can vectorize a loop. Otherwise it's meaningless to do analysis as the vectorization on interleaved accesses is unsafe.
The analysis collects interleave groups and records the relationships between the member and the group in a map.
Definition at line 388 of file VectorUtils.h.
|
inline |
Definition at line 390 of file VectorUtils.h.
|
inline |
Definition at line 395 of file VectorUtils.h.
| void InterleavedAccessInfo::analyzeInterleaving | ( | bool | EnableMaskedInterleavedGroup | ) |
Analyze the interleaved accesses and collect them in interleave groups.
Substitute symbolic strides using Strides. Consider also predicated loads/stores in the analysis if EnableMaskedInterleavedGroup is true.
Definition at line 799 of file VectorUtils.cpp.
References B, llvm::dbgs(), llvm::dyn_cast(), E, llvm::MapVector< KeyT, ValueT, MapType, VectorType >::empty(), llvm::SCEVConstant::getAPInt(), llvm::InterleaveGroup< InstTy >::getIndex(), llvm::getLoadStoreAddressSpace(), llvm::getLoadStorePointerOperand(), llvm::Instruction::getParent(), llvm::getPtrStride(), llvm::APInt::getSExtValue(), llvm::SetVector< T, SmallVector< T, N >, SmallDenseSet< T, N > >::insert(), llvm::InterleaveGroup< InstTy >::insertMember(), llvm::HexagonMCInstrInfo::isPredicated(), LLVM_DEBUG, llvm::Instruction::mayReadFromMemory(), llvm::Instruction::mayWriteToMemory(), llvm::MapVector< KeyT, ValueT, MapType, VectorType >::rbegin(), llvm::SetVector< T, SmallVector< T, N >, SmallDenseSet< T, N > >::remove(), llvm::MapVector< KeyT, ValueT, MapType, VectorType >::rend(), and llvm::InterleaveGroup< InstTy >::setInsertPos().
|
inline |
Get the interleave group that Instr belongs to.
Definition at line 428 of file VectorUtils.h.
Referenced by llvm::VPValue::replaceAllUsesWith().
|
inline |
Definition at line 435 of file VectorUtils.h.
References llvm::make_range().
| void InterleavedAccessInfo::invalidateGroupsRequiringScalarEpilogue | ( | ) |
Invalidate groups that require a scalar epilogue (due to gaps).
This can happen when optimizing for size forbids a scalar epilogue, and the gap cannot be filtered by masking the load/store.
Definition at line 1035 of file VectorUtils.cpp.
References llvm::dbgs(), I, llvm::SmallPtrSetImpl< PtrType >::insert(), LLVM_DEBUG, and llvm::InterleaveGroup< InstTy >::requiresScalarEpilogue().
|
inline |
Check if Instr belongs to any interleave group.
Definition at line 420 of file VectorUtils.h.
|
inline |
Returns true if an interleaved group that may access memory out-of-bounds requires a scalar epilogue iteration for correctness.
Definition at line 441 of file VectorUtils.h.
References llvm::AMDGPU::HSAMD::Kernel::Arg::Key::Align, assert(), B, llvm::LoopAccessInfo::blockNeedsPredication(), llvm::DenseMapBase< DenseMap< KeyT, ValueT, KeyInfoT, BucketT >, KeyT, ValueT, KeyInfoT, BucketT >::count(), llvm::DenseMapBase< DenseMap< KeyT, ValueT, KeyInfoT, BucketT >, KeyT, ValueT, KeyInfoT, BucketT >::end(), llvm::DenseMapBase< DenseMap< KeyT, ValueT, KeyInfoT, BucketT >, KeyT, ValueT, KeyInfoT, BucketT >::erase(), llvm::SmallPtrSetImpl< PtrType >::erase(), llvm::DenseMapBase< DenseMap< KeyT, ValueT, KeyInfoT, BucketT >, KeyT, ValueT, KeyInfoT, BucketT >::find(), llvm::LoopAccessInfo::getDepChecker(), llvm::MemoryDepChecker::getDependences(), llvm::InterleaveGroup< InstTy >::getFactor(), llvm::InterleaveGroup< InstTy >::getMember(), llvm::DenseMapBase< DenseMap< KeyT, ValueT, KeyInfoT, BucketT >, KeyT, ValueT, KeyInfoT, BucketT >::insert(), llvm::SmallPtrSetImpl< PtrType >::insert(), llvm::HexagonMCInstrInfo::isPredicated(), llvm::DenseMapBase< DenseMap< KeyT, ValueT, KeyInfoT, BucketT >, KeyT, ValueT, KeyInfoT, BucketT >::lookup(), llvm::cl::Sink, and Size.
|
inline |
Invalidate groups, e.g., in case all blocks in loop will be predicated contrary to original assumption.
Although we currently prevent group formation for predicated accesses, we may be able to relax this limitation in the future once we handle more complicated blocks.
Definition at line 407 of file VectorUtils.h.
References I, and llvm::SmallPtrSetImpl< PtrType >::insert().
1.8.13